A home products manufacturing company was working to develop custom solutions around their products by integrating delivery of data and insights that would improve operational efficiency for their customers.
As an industry leader and innovator in facilities control and security products, the company needed to provide functional solutions in a timely and efficient fashion.
To this end, the company was collecting more than 3TBs of IoT data from over 5 million deployed devices daily, and needed to create an environment in which they could store and analyze these large volumes of data.
With so much data being collected from so many sources – including web applications, mobile applications, and IoT sensors within product devices – the company encountered numerous roadblocks to success:
- Connection and ingestion were inefficient
- Slow response times inhibited the ability to deliver meaningful analytics in time to impact customers’ operational efficiency
- Data aggregation constraints made near real-time analytics impossible
- The existing environment was unstable, unreliable, and did not meet the company’s service level agreements
- Traditional storage and SQL database technologies were cost prohibitive, and could not be used within project budgets
The company selected Vertical Trail as a technology partner because of its experience helping clients rapidly design and establish analytics Innovation Labs and Big Data environments that leverage new, lower cost, open source technologies.
“Vertical Trail was able to offer a wealth of expertise in designing and managing Big Data environments,” says Managing Partner Larry Pollastrini. “We also have in-depth experience using Hadoop distribution technologies to support advanced analytics platforms. Overall, our team’s skills aligned very well with our client’s goals.”
The customer wanted a way to stream data in near real-time from millions of reporting devices – coupled with data from existing applications – to a centrally managed data hub where it could be immediately leveraged to deliver high-value insights. It was also important to capture and store the data in a way that would allow future analysis and discovery of new insights that could, in turn, be used to create new solutions.
Vertical Trail first conducted a one-week analysis of the existing environment to understand the current issues and create a new, modified design and prioritized action plan to…
- Stabalize the current environment, and then
- Move to the new, desired state.
Rapid Insights in Real Time
Working in collaboration with the customer, Vertical Trail took only six weeks to design and implement a new big data solution that leveraged Cloudera, and was deployed on cloud-based services from Amazon Web Services (AWS).
All data was consolidated and moved into S3, and made available for analysis in an innovation environment that leverages Python and R to discover new insights.
As the analytics are approved and operationalized, they are delivered through visualization tools, including Tableau. Data over 90 days old is archived in AWS Glacier, and stored for future analysis as new use cases are defined.
With the new solution in place, analytics were delivered within 3 minutes of collection from sensors embedded in product devices. Compare this to the previous solution, which took a minimum of 6-8 hours to deliver any insights. The data is now so current that the company’s customers can implement operational changes on a near real-time basis. This improvement resulted in a major increase in customer interest, as well as increased implementation of the company’s solutions.
The company will now be able to attract new types of customers, and expand its current product offerings to include complete, packaged solutions that positively impact operational efficiency.
The net result is an anticipated 20% annual growth – in an industry where growth is traditionally expected to be about 4-5%.
Vertical Trail understands how to manage large volumes of data in support of advanced analytics. Our ability to design and deploy complete solutions – and our expertise in leveraging new technology options such as AWS – helps our clients improve performance, expand offerings, and accelerate growth.