Big data has irrevocably changed the digital landscape, opening new doors of opportunities for businesses to deliver highly personalized customer experiences, streamline external and internal operations, and increase their revenue streams. While harnessing big data could be a game changer for businesses, the exponentially increasing growth of data can also pose a number of challenges that need to be carefully and successfully managed and navigated.
To help IT teams address the complexity of managing large volumes of data, WhereScape, the leader in data infrastructure automation software, offers a robust and reliable automation platform that helps simplify, accelerate, and automate the flow of big data, from staging to reporting.
SourceForge recently spoke with Neil Barton, CTO at WhereScape, to discuss the importance of data infrastructure automation for modern, data-driven businesses. Barton also details how WhereScape empowers IT teams by accelerating data warehouse development and delivering quick time-to-results while saving time and reducing costs.
Q: Please tell us a brief overview of WhereScape (year founded, size, solutions, industries you serve, etc.)? What problems does WhereScape seek to solve?
A: WhereScape helps information technology teams and organizations of all sizes to leverage automation to design, develop, deploy, and operate data infrastructure faster. Founded in 2001, WhereScape has more than 700 customers worldwide who rely on the automation technology to eliminate hand-coding and other repetitive, time-intensive aspects of data infrastructure projects to deliver data warehouses, vaults, lakes, and marts in days or weeks rather than in months or years.
Neil Barton, CTO at WhereScape
Q: Data is growing at a rapid pace and doubling in size every two years. According to the IDC, by 2020, the accumulated volume of big data will increase from 4.4 zettabytes to approximately 44 trillion gigabytes. In your opinion, what challenges do today’s enterprises face because of the exponentially increasing growth of data?
A: Today’s businesses are surrounded by data growing rapidly from a wide array of sources— from operational data to sensor data, web interaction, to mobile data, both static and streaming data. This presents challenges in both scale and type – and organizations need to be ready to handle this. With the right infrastructure and analysis, this breadth and depth of information can help companies better understand and engage with customers, take advantage of new market opportunities, optimize operations, and much more.
As more business decisions become driven by data, the greater the pressure on IT to deliver high-value data infrastructure and big data integration projects quickly. That’s why more IT teams are turning to agile data warehouse automation to provide the speed, agility, quality, and consistency that today’s business users demand.
Q: What is data streaming and how can this empower today’s IT organizations?
A: Unlike traditional batch-based data flows, where data is loaded in sets on a regular basis, such as hourly, daily or monthly — real-time or streaming data flows are often more time-sensitive to analyze, consist of larger volumes of data, and may or may not be of long-term value to persist or store within a company’s data landscape after processing.
The automation of streaming data flow enables IT teams to efficiently and cost-effectively leverage real-time streaming data from IoT, social media, and other streaming data sources within their data infrastructure. Through streaming data automation, organizations gain new possibilities for data science, machine learning, anomaly detection, and real-time analytics for organizations that have been traditionally been limited to batch-based data sources, processing, and analysis.
Q: As the global leader in data infrastructure automation, how can current approaches to data streaming help today’s IT organizations improve the way information architectures and software applications are being developed?
A: WhereScape automation with Streaming allows the delivery of successful projects to be completed seamlessly and within the reach of individual developers in days rather than months. Additionally, streaming data automation handles the complexity of working with streaming data, new sources and new data platforms for developers who may have limited experience working within this area, ensuring best practices are implemented.
By automating the repetitive, time-intensive tasks associated with design, development, deployment, and operation of data infrastructure – inclusive of both streaming and non-streaming data – organizations can increase development team productivity all while cutting down project timeframes and reducing the overall cost to the business.
Q: Can you tell us a bit more about WhereScape Automation Software? What are its key features and capabilities?
A: Whether organizations are tackling the creation and operation of data warehouses, data marts, data vaults, or big data integration projects, WhereScape automation software helps businesses provide data and analytics to the business 80 percent faster, with less cost and risk. From data infrastructure design and development through to deployment and operation, WhereScape automation makes it easier for IT teams to support their business with solutions that are primed and ready to quickly respond to change.
Q: In what ways can WhereScape automation with Streaming reduce the learning curve for IT teams?
A: WhereScape automation with Streaming supports IT teams by addressing the complexity of managing both traditional batch-based data and streaming real-time data as they design, create, deploy, and operate their data infrastructure, speeding up data delivery, and providing new analytics opportunities. 
It allows IT teams to automate the repetitive, time-intensive aspects of development projects in order to focus on higher value activities, automatically documenting each step along the way for future reference. Lastly, it flattens the learning curve of streaming data ingestion and is used for organizations lacking development experience in this area.
In addition, WhereScape automation with Streaming helps reduce the learning curve by easing both the connectivity to relevant source and target platforms as well as providing a simple-to-use graphical interface to define data flows rather than requiring developers to write complex Java, Scala, and Python (or similar) code. The tool also provides built-in best practices to help teams develop consistent, repeatable, and efficient data pipelines.
Q: What makes WhereScape stand out from other data automation and streaming platforms in the market?
A: WhereScape automation for Streaming has an advantage in the market because we remain laser-focused on the time to value for our customers. We don’t just ingest data, we automate the processes involved in preparing it for analysis. Our solution is also very comprehensive, addressing the reality that virtually every organization will have both streaming and non-streaming data to consider in their infrastructure planning and development.
WhereScape stands out from other tools in that we handle both the streaming and traditional batch-based processing, allowing teams to blend data as appropriate, along with providing full lifecycle development automation, including automated documentation and lineage.
Q: Looking forward, what challenges do you see to the data automation and streaming industries? How is WhereScape striving to meet these challenges?
A: As technology advances, the challenge will be the major increase in data that businesses will hold and be exposed to. Automation in data warehousing is vital, because it is the only way to keep pace with this data increase, and eliminates the hand-coding and repetitive, time-intensive aspects of data infrastructure projects. 
Automation ensures new data infrastructure can be delivered and evolved to meet changing business needs in days, rather than months or years. Secondly, it frees up IT teams to concentrate on the more strategic work of analysis and deriving insights that drive real business value. With WhereScape, we’re constantly working to make sure our automation offering takes into consideration the changing landscape of new types, sources and uses of data, such as with streaming data, to keep organizations ready to take advantage of the new data analytics opportunities rapidly coming their way.
As streaming technologies advance and change at a rapid pace, this presents inherent challenges for any organization. How do they choose the right technologies? How do they evolve/replace the technologies over time? Automation tools, and WhereScape, in particular, provides companies the ability to handle the ever-evolving technology landscape by automating the processing infrastructure. This reduces the risk, time, and cost when a company wants to change portions of their infrastructure over time.
Q: What’s next for WhereScape? Are you brewing up any new offerings for your customers?
A: At WhereScape, we continue to evolve the automation capabilities we offer our customers to ensure they are ready to take advantage of next-gen analytics opportunities such as machine learning, AI, anomaly detection and more. In an era of increasing data complexity, it is our mission to help organizations overcome the challenges of data on a large scale and reduce the time needed to deliver real business value. WhereScape’s focus is on the entire development lifecycle and we are always looking at ways to apply automation to reduce the time, cost, and risk of delivering data infrastructure.
About WhereScape
WhereScape helps IT organizations of all sizes leverage automation to design, develop, deploy, and operate data infrastructure faster. More than 700 customers worldwide rely on WhereScape automation to eliminate hand-coding and other repetitive, time-intensive aspects of data infrastructure projects to deliver data warehouses, vaults, lakes, and marts in days or weeks rather than in months or years. WhereScape has offices in Portland, Oregon; Reading, UK; Auckland, New Zealand; and Singapore.