IBM views Apache Spark as a critical technology for addressing key challenges and delivering the benefits of intelligence based, in-time action. As a result, IBM is making significant investments to support the growth of the core Spark project and its community, to help develop the skills of the next-generation data practitioner and to facilitate IBM solutions by incorporating Spark.
IBM is also committed to advancing the use of Spark so that a growing community of Spark-savvy data scientists and data analysts can quickly transform today’s problems into tomorrow’s solutions. IBM believes that transformation is essential to business and is committed to helping data scientists realize the benefits associated with those transformations.
Building Community For more than 40 years, IBM has demonstrated a strong commitment to the open source community, making instrumental contributions to Linux, Hadoop and other open source projects. IBM is a founding member of the Open Data Platform (ODP) Initiative, an association formed to facilitate collaboration, innovation and standardization for big data technologies. IBM has also been a key member of the Apache Spark community since the beginning. IBM is one of four founding members of AMPLab, the organization based at the University of California, Berkeley that created Apache Spark. AMPLab hosts training programs and will offer new courses developed by IBM and AMPLab faculty.
In the online world, IBM is launching a digital community for Spark and has created a Spark social badge for showcasing Spark expertise on social media.
IBM is opening the first Spark Technology Center, which will help foster development of the core technology, serve as a business incubator and help promote Spark in the technology industry. The Spark Technology Center is located at the San Francisco campus of Galvanize, an organization dedicated to enabling new growth opportunities through industry-focused education. Each Galvanize campus brings together industry partners, individual data scientists, academic researchers, students and others all under one roof. The purpose of the Spark Technology Center is to create and promote open and free educational assets for enabling Spark adoption and sharing best practices, as well as evangelize Spark technology and its business potential.
IBM is partnering with organizations to help its clients apply Apache Spark to a wide range of solutions. For example, IBM has partnered with AMPLab, DataCamp, MetiStream, Galvanize and Big Data University. IBM’s goal is to help build a community, deliver best-of-breed training, create innovation labs and produce a reference architecture to speed time to value.
Unifying analytics on and around Spark
IBM is incorporating Spark into its big data and analytics solutions, such as IBM® BigInsights® for Apache Hadoop and the IBM Bluemix™ cloud platform. Other IBM solutions also complement Spark. For example, IBM Streams improves on Spark’s built-in streaming capabilities, enabling streaming analytics for cybersecurity, financial trading and other applications that require extremely low latency.