As users constantly access, store, and analyze more and more data, and as the demand for faster speeds increases, In-memory Computing (IMC) has become a popular process that businesses are pursuing. The technology involves keeping and storing data in the server’s main random access memory (RAM) instead of in complex relational databases running on relatively slow disk drives. Global technology researcher and advisory company Gartner predicts that the adoption of IMC technologies will continue to rise because it provides a cost-effective path to creating modern, scalable, real-time applications. Additionally, the analyst firm predicts that by 2018, the market for in-memory data grids will reach $1 billion.
SourceForge spoke with Terry Erisman, the Vice President of Marketing for GridGain Systems, to understand the benefits and importance of in-memory computing for the modern enterprise, and also explore how GridGain is revolutionizing real-time data access with their enterprise-grade in-memory computing solutions.
Q: Can you please tell us more about your company, including a brief history, the year founded, company size, solutions you offer, and the industries you serve?
A: GridGain Systems was founded in 2007 and currently has around 115 employees. GridGain has averaged triple-digit annual sales growth over the past four years and nearly tripled prior year sales in 2016. GridGain was recognized as #187 on the 2017 Inc. 500 and named the #2 fastest growing private company on the Silicon Valley Business Journal Fast 100.
Terry Erisman, the Vice President of Marketing at GridGain
GridGain Systems offers enterprise-grade in-memory computing solutions built on Apache® Ignite™. GridGain solutions are used by global enterprises in financial services, fintech, software, ecommerce, retail, online business services, healthcare, telecom and other major sectors, with a client list that includes Barclays, ING, Sberbank, Finastra, IHS Markit, Workday and Huawei. GridGain solutions connect data stores (SQL, NoSQL and Apache® Hadoop®) with web-scale applications and enable massive data throughput and ultra-low latencies across a scalable, distributed cluster of commodity servers. GridGain is the most comprehensive, enterprise-grade in-memory computing platform for high-volume ACID transactions, real-time analytics, and hybrid transactional/analytical processing (HTAP).
As an open source software and services company, GridGain Systems generates revenue from annual GridGain Professional, Enterprise, and Ultimate Edition support subscriptions, professional services, and the annual In-Memory Computing Summit conference series.
Q: For individuals that are unfamiliar with In-Memory Computing (IMC), can you offer a brief overview and description in the simplest of terms? What are its advantages over disk-based architectures?
A: In-memory computing (IMC) uses large pools of RAM to process and analyze data without the need to continually read and write data in a disk-based database. Today’s in-memory computing platforms typically take advantage of a distributed architecture for parallel processing and leverage commodity servers to provide easy, low-cost scalability at any time simply by adding nodes to the cluster. These platforms can easily be inserted between existing application and database layers to deliver performance gains of 1,000x or more without rip-and-replace of the existing database. The distributed architecture of the IMC platform replicates data across the cluster nodes, providing high availability and simplified maintenance. In-memory computing platforms typically offer an:
- In-memory data grid to cache data and accelerate and scale out applications running on RDBMS, NoSQL, or Hadoop databases
- In-memory database that serves as the system of record while providing full relational database functionality
- Streaming analytics engine for analyzing and responding to incoming data in real-time
- The most advanced systems also offer ANSI-99 SQL support including DML and DDL
Q: Digital transformation has moved beyond the hype and has become a challenge that today’s modern enterprises must tackle head-on. So how can IMC help enterprises address digital transformation?
A: Digital transformation initiatives, such as web-scale applications and IoT use cases, require that businesses create modern, scalable, real-time applications. They must handle massive data growth from a variety of sources, provide interactive access to that data often in real-time, scale for access over the internet, and offer real-time situational awareness. To meet these demands, businesses must achieve unprecedented levels of performance and scale.
In-memory computing can provide a 1,000x increase in performance for existing applications, enabling organizations to launch their digital transformation initiatives with minimal investment and disruption – no rip-and-replace – while achieving unprecedented speed and scale. As described in keynotes at the recent In-Memory Computing Summit North America, for example:
- Sberbank is building an in-memory computing architecture that will rival the capabilities of today’s largest supercomputers
- Wellington Management is managing $1 trillion in assets on their in-memory computing-driven Investment Book of Record (IBOR)
- Workday is processing 189 million transactions a day from 26 million workers under contract on an in-memory computing platform
In-memory computing can also allow organizations to begin developing a strategy around Hybrid Transactional/Analytical Processing (HTAP), which provides the ability to process transactions and perform real-time analytics on the operational dataset. HTAP enables real-time analytics and situational awareness on live transaction data as opposed to after-the-fact analysis on stale data. Gartner has said that IMC-enabled HTAP “can have a transformational impact on the business.”
Whether the digital transformation initiative involves machine learning and AI, Internet of Things, blockchain, or virtual or augmented reality, in-memory will be essential to the success of these initiatives as they scale.
Q: Not all data grid products are created equal. So what features and factors should organizations consider when moving to IMC from disk-based architectures?
A: Key factors to consider include the following:
- Minimal disruption – The solution should not require IT to rip-and-replace existing systems. This is simply too costly and time-consuming – and it’s unnecessary.
- Open source software with enterprise-grade support – Open source is where the real IMC innovation is occurring, with very active communities. The open source model also has the advantage of not trying to leverage and continue to cash in on a vendor’s legacy solutions. This invariably leads to faster innovation at lower cost. Still open source by itself is usually not sufficient for enterprises to run software in production systems. The open source software should be supported by a company that offers enterprise-grade add-on features and support and that is willing to develop a deep understanding of the customer’s infrastructure to be able to offer specific guidance on how best to take advantage of IMC capabilities, such as how to distribute the data across the in-memory computing cluster.
- No hardware lock-in – Because of the rapid pace of change – and to save money – organizations should look for a solution that leverages commodity hardware, eliminating vendor lock-in.
- Comprehensive solution – As much as possible, the solution should meet all of the organization’s in-memory computing platform needs both today and tomorrow, eliminating the time and cost of integration projects.
- Ability to start small and grow – From both a hardware and software perspective, the solution should enable enterprises to start small and then easily and cost-effectively scale the in-memory computing RAM and CPU pool.
Q: Can you tell us a bit more about GridGains’s In-Memory Computing Platform? What are its highlights, and what separates it from the other similar options in the market?
A: GridGain has the following distinguishing features and capabilities:
- A unified high-performance architecture – The GridGain in-memory computing platform consists of multiple features connected by a clustered, in-memory file system. The In-memory Data Grid, In-memory Database, Streaming Analytics and Service Grid are interconnected. Computations occur as close as possible to the data. Additional features such as high throughput, low latency, load balancing, caching, in-memory indexing, streaming, Hadoop acceleration and other performance improvements are crucial to success in real-time modeling, processing, and analytics.
- Scalability – GridGain is massively scalable up to petabytes of in-memory data, allowing companies to add cluster nodes and memory in real-time with automatic data rebalancing. Because it is hardware-agnostic, users can choose their own preferred hardware for scaling up. Scaling up and down can also occur dynamically across cluster nodes located in multiple hosted datacenters, enabling enterprises to cost-effectively handle spikes in performance.
- Full SQL support – GridGain is ANSI SQL-99 compliant and the Distributed SQL capabilities support DML and DDL, so users can leverage their existing SQL code using the GridGain JDBC and ODBC APIs. Users with existing code bases that are not based on SQL can leverage their existing code through supported APIs for Java, .NET, C++, and more.
- A Distributed ACID and ANSI-99 SQL-Compliant Disk Store. The optional GridGain Persistent Store is a distributed ACID and ANSI-99 SQL-compliant disk store available in Apache Ignite. It may be deployed on spinning disks, solid state drives (SSDs), Flash, 3D XPoint or other similar storage technologies. If used, the optional Persistent Store keeps the superset of data and all the SQL indexes on disk, which allows GridGain to be fully operational from disk. The combination of this new feature and the platform’s advanced SQL capabilities allows GridGain to serve as a distributed transactional SQL database, spanning both memory and disk, while continuing to support all of the traditional in-memory only use cases.
- High availability. The GridGain in-memory computing platform provides essential high-availability features such as data-center replication, automatic failover, fault tolerance, and quick recovery on an enterprise-level scale.
- Transaction processing. The GridGain platform supports ACID-compliant transactions in a number of user-configurable modes.
- Security features. GridGain supports authentication, authorization, multiple encryption levels, tracing, and auditing.
- Open Source framework. GridGain is based on Apache® Ignite™, an Apache Software Foundation open source project. GridGain Systems was the original creator of the code contributed to the Apache Software Foundation that became Apache Ignite and fully supports the technology behind Apache Ignite. The GridGain Enterprise Edition extends the features in Apache Ignite to provide enterprise-level capabilities and services, such as additional security, data center replication, rolling production updates, auditing mechanisms, a GUI for management and monitoring, network segmentation, and a recoverable local store. The GridGain Ultimate Edition includes all the features of the GridGain Enterprise Edition plus a Cluster Snapshots feature for automated backups when using the GridGain Persistent Store feature in production environments.
- Production Support. GridGain Systems Support, available to GridGain Professional, Enterprise, and Ultimate Edition users, includes rolling updates, faster availability of all releases and patches, and 24/7 enterprise-level support.
Q: GridGain’s IMC platform is built using an open source model. Given this, how do you believe businesses can benefit from utilizing an open source IMC platform?
A: Open source is where in-memory computing innovation is occurring, with very active communities contributing to the projects. Companies that leverage open source-based solutions benefit from the contributions of all community members. In addition, companies evaluating open source in-memory computing solutions can deploy and test the solutions on their own at any time, allowing them to become familiar with the open source capabilities before engaging with a commercial vendor. This invariably leads to faster adoption at a lower cost. By selecting an open source IMC platform project with a strong commercial company behind it, companies can ensure that help will be available when they need it, if they hit any speed bumps in their implementation process.
Q: What would you say are the best use cases for IMC?
A: Almost any data-intensive environment can benefit from in-memory computing. The technology continues to enjoy broad adoption and interest within the financial services and fintech industries, which were some of the earliest adopters. Interest has grown in digital transformation initiatives such as web-scale online business services and IoT platforms. In addition, many uses have emerged among healthcare, retail and telecommunications companies. HTAP use cases in which companies need to transact and analyze on their operational dataset in real-time are rapidly growing.
Q: This year, IMC has continued to grow at a rapid rate. What would you say are the major factors that contribute to this increased demand? And what message or advice would you like to share with organizations that have yet to consider IMC as a viable option?
A: Extreme data growth is continuing, from 8ZB in 2015 to 35ZB in 2020. At the same time, the cost of memory continues to fall, about 10 percent per year. Many organizations now recognize that the performance gains from in-memory computing justify the additional cost of an IMC platform.
This makes in-memory computing the most practical and cost-effective solution for today’s critical fast data need. CEOs and boards are demanding digital transformation to meet evolving customer and employee expectation – many initiatives will stumble and fail at scale without IMC. In-memory computing platforms offer huge performance gains with relatively low investment and minimal disruption as companies unlock the data available to them to better serve their end users. Further, in-memory data grids are entering mainstream adoption. As a result, they are being used across a broader range of industries. This trend will continue as more organizations see the value of extreme speed and scale and the cost/benefit tradeoff continues to improve.
Q: What rising trends do you think will affect the future of IMC? How is GridGain meeting these head on?
A: The trends noted above, including digital transformation initiatives such as web-scale applications, IoT, AI/machine learning, and the use of HTAP will continue for years, and we are moving toward a point where all data will be in-memory and disk storage will be used primarily for backup and archiving. GridGain is continuing to develop solutions ahead of market needs and will support all these trends.
About GridGain Systems
GridGain Systems is a trusted provider of enterprise-grade in-memory computing solutions built on Apache® Ignite™. Established in 2007, the company aims to address today’s fast-data challenges for high-volume transactions, real-time analytics and hybrid transactional/analytical processing (HTAP) with the GridGain In-Memory Computing Platform. Today, GridGain is trusted by global enterprises across various sectors, including financial, telecom, software, online business services and more.