12 Major Mistake Companies Make With #BigData

HuffPost is not normally the go-to for business information but this is great online piece that asks 12 members of Young Entrepreneur Council (YEC) their opinion on the top mistake companies make with big data.  I’ll list a few below and the link to the article is at the bottom.

CPG companies working with retailer supplied data should make of note of this.


Not Making Data-Driven Decisions

Data equals power.

“I believe that most companies don’t realize how much you can pull out of your data. There are many tools out there that can help you make data-driven decisions, which in turn can give you more predictable results.” – Elliot Bohm,


Not Having Data Scientists

“Collecting big data is easier then ever and implementing tools to work with big data has also become much more accessible. The problem is oftentimes companies do not have a qualified Data Scientist or someone who can interpret or map/reduce the proper dimensions of data. Instead they rely on non-qualified personnel to interpret data. The improper analysis of data can be very harmful to a company.” – Phil Chen, Systems Watch

OK – but you can focus on what you do best by hiring a third-party BI firm to handle this – contact us for a complimentary consultation.


Answering Trivial Questions With It

“The biggest mistake that companies make with big data is using it to answer relatively trivial questions such as “what.” Big data isn’t about “what” questions; it’s about “why” questions. Big data is about joining data sets that have never been joined before and asking questions that have never been asked. It’s about knowing why customers and employees are doing the things that they do.” – Dusty Wunderlich, Bristlecone Holdings


Focusing on Data Processing at the Expense of Analysis

“Half of the challenge of big data is finding the right algorithms and approaches to ingest the vast quantities of information you have. The second, and more overlooked, challenge is finding a way to present your findings in a usable fashion. Too many companies focus on the former (how do we process all that data?) at the expense of the latter (how do we make it actionable?).” – AJ Shankar, Everlaw, Inc.


Confusing Correlation and Causation

“When companies work with big data, a major and common mistake is to assume that correlation implies causation. While you can use data to understand correlation, equating it to “cause and effect” can lead to false results and fruitless decisions. Making the distinction between correlation and root cause is critical to utilizing data for best results.” – Doreen Bloch, Poshly Inc.


Not Using It to Answer Business Questions

“There’s so much data being generated and collected, it can be overwhelming. Successful organizations start with the business questions they want to answer and then assess the data they have to answer those questions. Just looking at your mountain of data and trying to figure out what to do with it is a recipe for a lot of wasted time and effort.” – David Booth, Cardinal Path


There are additional ideas at the online article.


Learn how to put big data insights to work for you by contacting us for a complimentary consultation.



Source: 12 Major Mistake Companies Make With Big Data






#Walmart: The Big Data Skills Crisis And Recruiting #BusinessAnalytics Talent

As the amount of digital information generated by businesses and organizations continues to grow exponentially, a challenge –or as some have put it, a crisis–has developed.

There just aren’t enough people with the required skills to analyze and interpret this information–transforming it from raw numerical (or other) data into actionable insights – the ultimate aim of any Big Data-driven initiative.

One survey recently carried out by researchers at Gartner IT +0.93% found that more than half of the business leaders they queried felt their ability to carry out analytics was restricted by the difficulty in finding the right talent.

Overcoming this problem is a challenge that all companies will have to face, and market leaders–aware that they have more to lose than many by falling behind in the race to keep up with technology–have come up with some innovative solutions.

Walmart decided to apply one of the fundamental weapons in the Big Data arsenal–crowdsourcing–to the problem, with positive results.

Last year, they turned to crowdsourced analytics competition platform Kaggle. At Kaggle, an army of “armchair data scientists” apply their skills to analytical problems submitted by companies, with the designer of the best solution being rewarded – sometimes financially, in this case with a job.

To continue reading click the source link below.


Source: Walmart: The Big Data Skills Crisis And Recruiting Analytics Talent











27 Free Books on #DataMining


Part of our mission statement is to make CPG reports that strive for a higher level of insights. Consider the 8 levels of analytics below as coined from SAS. Where does your organization stand on this list?

1) Standard reports – Standard reports provide summary statistics and answer questions like “What happened?” and “When did it happen?” said Davis. “That’s analytics, but not enough.”

2) Ad hoc reports – Ad-hoc reports answer questions like, “How many? How often? Where?” he said. They provide a level of independence on desktops that allow an individual, for example, to see sales in a particular region or at a particular point in time without needing to go to an IT governance counsel and wait three months for the result.

3) Query drill-downs – Also referred to as OLAP, query drill-downs answer questions like, “Where exactly is the problem?” and “How do I find the answers?” said Davis. This is for when an organization wants to see not only the results, but what the results mean and what backs it up, he explained.

4) Alerts – Alerts answer questions like, “When should I react?” and “What actions are needed now?” said Davis. “This is when you reach a particular threshold … something changes from green to red, so you do something about it.”

5) Statistical analysis – Statistical analysis answers the questions, “Why is this happening?” and “What opportunities am I missing?” he said. “You begin to take the data … and you begin to understand why things are happening.”

6) Forecasting – A popular level, forecasting answers questions like, “What if these trends continue? How much is needed? When will it be needed?” he said.

7) Predictive modeling – Predictive modeling tells users what will happen next and how it will affect the business, Davis said.

8) Optimization – Optimization answers the questions, “How do we do things better?” and “What is the best decision for a complex problem?


Want to read up on how to make it to level 8? We’ve found a list of 27 free machine learning (think artificial intelligence, data mining, statistical inference, predictive modeling) books to whet your appetite.

Don’t pull a muscle! For a free consultation on your business intelligence strategy contact us.

  1. An Introduction to Statistical Learning: with Applications in R
    Overview of statistical learning based on large datasets of information. The exploratory techniques of the data are discussed using the R programming language.
  2. Data Science for Business: What you need to know about data mining and data-analytic thinking
    An introduction to data sciences principles and theory, explaining the necessary analytical thinking to approach these kind of problems. It discusses various data mining techniques to explore information.
  3. Modeling With Data
    This book focus some processes to solve analytical problems applied to data. In particular explains you the theory to create tools for exploring big datasets of information.
  4. Big Data, Data Mining, and Machine Learning: Value Creation for Business Leaders and Practitioners
    On this resource the reality of big data is explored, and its benefits, from the marketing point of view. It also explains how to store these kind of data and algorithms to process it, based on data mining and machine learning.
  5. Data Mining: Practical Machine Learning Tools and Techniques
    Full of real world situations where machine learning tools are applied, this is a practical book which provides you the knowledge and hability to master the whole process of machine learning.
  6. Machine Learning – Wikipedia Guide
    A great resource provided by Wikipedia assembling a lot of machine learning in a simple, yet very useful and complete guide.
  7. Data Mining and Analysis: Fundamental Concepts and Algorithms
    A great cover of the data mining exploratory algorithms and machine learning processes. These explanations are complemented by some statistical analysis.
  8. Mining the Social Web: Data Mining Facebook, Twitter, LinkedIn, Google+, GitHub, and More
    The exploration of social web data is explained on this book. Data capture from the social media apps, it’s manipulation and the final visualization tools are the focus of this resource.
  9. Probabilistic Programming & Bayesian Methods for Hackers
    A book about bayesian networks that provide capabilities to solve very complex problems. Also discusses programming implementations on the Python language.
  10. Data Mining Techniques: For Marketing, Sales, and Customer Relationship Management
    A data mining book oriented specifically to marketing and business management. With great case studies in order to understand how to apply these techniques on the real world.
  11. Inductive Logic Programming Techniques and Applications
    An old book about inductive logic programming with great theoretical and practical information, referencing some important tools.
  12. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition
    This is a conceptual book in terms of data mining and prediction from a statistical point of view. Covers many machine learning subjects too.
  13. An Introduction to Data Science
    An introductory level resource developed by a american university that presents a overview of the most important data science’s notions.
  14. Mining of Massive Datasets
    The main focus of this book is to provide the necessary tools and knowledge to manage, manipulate and consume large chunks of information into databases.
  15. A Programmer’s Guide to Data Mining
    A guide through data mining concepts in a programming point of view. It provides several hands-on problems to practice and test the subjects taught on this online book.
  16. Data Mining with Rattle and R: The Art of Excavating Data for Knowledge Discovery
    The objective of this book is to provide you lots of information  on data manipulation. It focus on the Rattle toolkit and the R language to demonstrate the implementation of these techniques.
  17. Reinforcement Learning: An introduction
    A solid approach to the reinforcement learning thematic providing solution methods. It describes also some very important case studies.
  18. Pattern Recognition and Machine Learning (Information Science and Statistics)
    This book presents you a lot of pattern recognition stuff based on the bayesian networks perspective. Many machine learning concepts are approached and exemplified.
  19. Machine Learning, Neural and Statistical Classification
    A good old book about statistical methodology, learning techniques and another important issues related to machine learning.
  20. Information Theory, Inference, and Learning Algorithms
    An interesting approach to information theory merged with the inference and learning concepts. This book taughts a lot of data mining techniques creating a bridge between it and information theory.
  21. Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die [Broken Link] A great predictive analytics book providing an insight about the concept, alongside with case studies to consolidate the theory.
  22. Introduction to Machine Learning
    A simple, yet very important book, to introduce everyone to the machine learning subject.
  23. Data Mining and Business Analytics with R
    Another R based book describing all processes and implementations to explore, transform and store information. It also focus on the concept of Business Analytics.
  24. Machine Learning
    A very complete book about the machine learning subject approching several specific, and very useful techniques.
  25. Think Bayes, Bayesian Statistics Made Simple
    A Python programming language approach to the bayesian statistical methods, where these techniques are applied to solve real-world problems and simulations.
  26. Bayesian Reasoning and Machine Learning
    Another bayesian book reference, this one focusing on applying it to machine learning algorithms and processes.  It is a hands-on resource, great to absorb all the knowledge in the book.
  27. Gaussian Processes for Machine Learning
    This is a theoretical book approaching learning algortihms based on probabilistic gaussian processes. It’s about supervised learning problems, describing models and solutions related to machine learning.


Original Source:




#DataAnalyticsTechnology: Turn insight into action with #PredictiveAnalytics


Good Saturday morning – hope you are enjoying your cup of coffee.  Now enjoy a -not too bad- geeky article on predictive analytics and how to make use of it.

See you again next week at the office.


Source: Turn insight into action with predictive analytics









Demystifying #SelfServiceData and the Future of #BusinessIntelligence

Imagine that you wanted to see a review of the latest weekend blockbuster, but you couldn’t just Google its Rotten Tomatoes score. Instead, you had to submit a written request to an information technology department, wait five days, then sift through a binder-sized report that breaks down the reviews by publication type, readership size, and reviewer age.

As fascinating as that information might be, it took five days too long — you went to the movie days ago on a hunch it would be good.

Welcome to the world of traditional business information.  This is the beginning of a look at self-service data analytics which is the future of business intelligence.

Continue reading the article from the link below and contact us for a free consultation on how to put self-service BI into the hands of those who need it the most – your category managers and sales analysts.
Source: Demystifying Self-Service Data and the Future of Business Intelligence





#DataForAnalytics: #PredictiveAnalytics tools point to better business actions

From recommending additional purchases based on the items that customers place in online shopping carts to pinpointing hospital patients who have a greater risk of readmission, the use of predictive analytics tools and techniques is enabling organizations to tap their collections of data to predict future business outcomes — if the process is managed properly.

Using predictive analytics tools lets organizations look ahead in an effort to optimize business strategies. But there has to be a purpose to the analytics efforts, and a solid plan behind them.

Predictive analytics has become an increasingly hot topic in analytics circles as more people realize that predictive modeling of customer behavior and business scenarios is “the big way to get big value out of data,” said Mike Gualtieri, an analyst at Forrester Research Inc. As a result, predictive analytics deployments are gaining momentum, according to Gualtieri, who said that he has seen an increase in adoption levels from about 20% in 2012 to “the mid- to high-30% range” now.

That’s still relatively low — which creates even bigger potential business benefits for organizations that have invested in predictive analytics software. If a company’s competitors aren’t doing predictive analytics, it has “a great opportunity to get ahead,” Gualtieri said.

Predictive analytics projects can also provide those benefits across various industries, said Eric King, president and founder of The Modeling Agency LLC, an analytics consulting and training services firm based in Pittsburgh. “Everyone is overwhelmed with data and starving for information,” King noted.

But that doesn’t mean it’s just a matter of rolling out the technology and letting analytics teams play around with data. When predictive analytics is done well, the business benefits can be substantial — but there are “some mainly strategic pitfalls” to watch out for, King said. “Many companies are doing analytics to do analytics, and they aren’t pursuing analytics that are measurable, purposeful, accountable and understandable by leadership.”

Data scientists don’t know it all

Plan ahead on predictive analytics

For a free consultation on your business intelligence strategy contact us.



Source: Predictive analytics tools point to better business actions









If you don’t really understand #bigdata how about tackling medium data first?

Excellent article about how big data is affecting all areas of our lives – link to the source is below.

The lifeblood of the information age is data and the prevailing wisdom is that the companies that can extract insights from data have an advantage over those that don’t.


The term “big data” refers to the huge quantities of raw data from outside the organization that can be commingled with internal data and mined for intelligence.

Analysis company Gartner says big data is

“high-volume, high-velocity and high-variety information assets that require cost-effective, innovative forms of information processing for enhanced insight and decision-making”.

Not necessarily, says Matt Kuperholz, a partner in PwC’s modelling and analytics group.

“Big data is simply using different tools and techniques to extract the full value from data.”


Businesses should tackle big data by asking themselves a business question, says Sally Wood, professor of business analytics at the University of Sydney Business School.

“I always say to businesses, ‘Tell me what you would do with the data if you had all the data in the world. What is the research question you want to answer?’ ”

Wood believes it’s the nature of the question that determines whether big data is the solution. So what sorts of questions require a big data solution?

Big data is also useful far beyond the realm of sales. “Some companies want to understand what factors affect leadership qualities,” Wood says. “All these things that were thought of as fluffy and non-rigorous suddenly can become much more evidence based.”


PwC’s Kuperholz says big data comes into its own when a business has a large number of customers who are serviced via multiple channels with different costs. Throw in serious competition to win those customers away and retain their loyalty and the case to use big data grows stronger.

“Then you have complex supply chains or complex processes that can be optimized,” he says. “How does [global freight and logistics company] UPS get a jump on lower cost of delivery? Because they optimize routes.”

In many US states, drivers can turn right through a red traffic light, so UPS puts more right-hand turns in its drivers’ routes. This has saved hundreds of millions of dollars. “That’s a clever use of analytics,” Kuperholz says.


Getting more from existing data wasn’t a reason for avoiding external big data among survey respondents. Of the 58 per cent of respondents not using big data, the reasons largely fell into two categories: cost and lack of understanding about its nature and benefits.

In fact, the advice is for businesses to make absolutely sure, before embracing big data, that they have a fantastic handle on the data they already generate.

“Most organizations have more than enough data to get started, but more need to know how to use it to drive commercial value,” says Sahil Merchant, head of McKinsey Digital Australia.

The challenge for companies is to develop their internal capabilities with their own data. “Rather than focusing on big data, companies can start with medium data and use what they have got.


Are we moving into an age when human instinct is redundant? The majority of those using big data still see greater value in the expertise of people (70 per cent rated the team as most important) over data (30 per cent). But that mindset is increasingly insufficient.


Start learning what insights you can gain from big data by contacting us for a free consultation.


Source: If you don’t really understand big data how about tackling medium data first?







#RetailAnalysis: Why #BigData Doesn’t Always Help

  • When big data is good and when big data is bad
    For thousands of years astronomers have gathered masses of data about the movements of the heavenly bodies, and have been highly effective at predicting where they are going to next. Why? Because, from our perspective, there are few disruptions to the passage of those stars and planets. Put it another way, big data is good at predicting, but only when things are predictable!
    Translating that into the world of big retail, the danger of big data is that can lull us into the believing in extrapolations. Extrapolate retail trends over the last few decades and you would see a trend towards one-stop shopping. Technology has disrupted that, and it would have been nigh on impossible for trend analysis out of retail loyalty card data to spot that disruption.
  • Too much granularity in the analysis of big data
    For many retailers, one of the major attractions of big data is its beautiful granularity. The ability to analyze down to the lowest level of transaction data is a dream come true to many. But too much focus ‘down in the weeds’ can lead a marketer to miss some of the bigger trends that more macro analysis might offer. For a retailer, obsessing about what happens in a particular store on a particular day may be interesting, but is dangerous if it is at the expense of broader trend analysis. This type of analysis inflates the value of short term deals such as promotions and could lead to their overuse.
  • Big data myopia
    One of the dangers of working for a data rich organization is that one forgets that the data you have isn’t the whole story. Loyalty card data is limited in two highly significant ways. Firstly, it is limited only to the shoppers who have a card (which might possibly skew towards those that are most loyal). If shoppers without cards begin to behave differently, that might not get spotted. Secondly, it only captures their transactions in your stores: missing the entirety of their relationship with your competitors.

You can gain significant insights into your business with big data – but only with someone who knows how to present that big data in meaningful ways – by answering the questions you want answered.

For a complimentary consultation please contact us.

Source: Why Big Data didn’t help Tesco


#BigData Adds to Availability of #CPG Analytics

The availability of more data feeding consumer analytics is rapidly increasing given the amount of data generated by consumer activity through web, mobile & social media.

No kidding Sherlock – you say – but how to turn that into actionable items is the real head scratcher.  Metric management and performance based management is probably not new.  Current data warehousing technologies are nothing but metric/performance based managements.  Most retail vendors and CPG companies have KPIs and dashboards that they use.  So this “new” big data revolution isn’t really new and it’s not really “bigger.”

The difference really is this – most retail suppliers are only using the POS and inventory data supplied by their retailer. Typically, this has gone into a structured data warehouse and, while predictable, is also inflexible.  Inflexible in the sense that it takes weeks or months of IT time to code a new data structure into a rigid data warehouse.

The real power of big data is that unstructured data can be brought into the analytics within days, sometimes hours. This allows for real ad hoc “what if” analysis. Flexible and powerful enough to let you ask – and get answered – questions that have percolated in the back of your mind.

  • How does weather temperature impact my sales?
  • How can I track consumer sentiment around a product launch and does that impact POS?

Ask your analytics provider if they can quickly bring you insights about data currently not in the data warehouse. Or do they simply provide you canned reports that everyone has seen a thousand times. Or worse, your provider will hardcode reports based on some archaic database language that requires a programmer dozens of hours (say expensive) to change.

When you are ready to see how to gain the insights provided by big data simply contact us and we bring you a demo of your new power source.





#CPG: 10 Reasons Why Now Is the Time to Get into #BigData

Rick Delgado is a new technologies freelance writer. In this article he takes an objective view of Big Data and lays out 10 reasons why now is the time for Big Data. My observations are italicized below.

1. It will keep your data secure

Being able to completely map your data out in front of you will make you better at analyzing potential threats. You can easily ensure that the most sensitive data you have in your system has all of the necessary security features it needs to stay well protected and within any sort of regulations that your business has to adhere to.
Many BI tools have provided for this sitting on top of the data – however – the big data structure allows this to be mapped into the clusters where data lives.

2. It opens up brand new revenue sources

You will gather a ton of insight into your part of the market. This insight is valuable, and not just to you. Selling out non-personalized industry data about trends in the market can bring in a ton of money from any other businesses operating in the same portion of the industry that you are, making it a great way to bring in some more funds.
While this is not always possible for a CPG company, for example, to do when retailer-sourced data is in the mix there are other opportunities at the enterprise level.

3. It’ll give you an advantage

Businesses are built on tradition. Any industry is going to have years of tradition that believe that one way of doing things is the right way and if it isn’t broke, don’t fix it. One important value that big data has is that it lets you really examine and analyze preconceived notions about aspects of a business you may have never even considered. It will provide better data when it comes to experimenting and trying to innovate, meaning that you’ll be able to get a much better advantage over your competitors.

4. Better Visuals

Big data requires cutting edge data visualizing tools in order to translate all of those numbers and data points into something a bit more tangible. This will increase overall usability that can be utilized by people in the business themselves or their end users.
There are increasing numbers of tools available for the user to create insight from their data lake – Vortisieze prefers MicroStrategy but ultimately is agnostic toward the BI tool selected giving you maximum flexibility.

5. It’s very easy to set up

Big data used to be something that only big corporations could afford to get into. Luckily, there are alternatives that can bring the benefits of big data to a company of any size. Hadoop is a great open source framework that can handle large amounts of data. Hadoop works with Excel, which most businesses already use.
The entry point is such, especially with the ABC of business (Analytics, Big Data, Cloud Computing), that retailer sales/support offices can easily add this as a budget item without involving their IT division.

6. It helps your business evolve

One of the best things about big data is just how scalable it is. Getting the right tools means being able to convert not just number data, but also text, audio, and video files to find patterns that can give a business a ton of insight into the market. Investing in big data as a small company means you’ll be able to easily handle all the data easily that comes with growth.
You begin seeing your business as your customers (the ones buying your products from the retailer) see your business.

7. It can significantly cut back on maintenance

Businesses that rely on large pieces of equipment often spend tons of money when it comes to replacing and maintaining them using predetermined data that tries to predict how long they’re going to remain efficient. Big data allows you to get rid of rounded averages and replace them with specifics, allowing you to squeeze even more life out of machinery that still has plenty of use left.
And with cloud computing, also called data-as-a-service, there is no equipment involved to maintain.

8. It brings you closer with your customers

The customers of today are a lot different than in the past. The rise of the internet allows them to thoroughly and tirelessly research a product before buying and to communicate with tons of people about which brand they should do business with. Big data allows you to better profile these fickle consumers and figure out specifically what they want.
See comments to #6 above.

9. It’s easier to analyze risk

There’s more to business than running your own company effectively. Every business is just one in a large industry that must regularly compete and innovate in order to stay ahead of the rest. Utilizing big data to analyze things like news articles and social media will give you cutting edge information about the biggest and latest developments in your industry.

10. You’ll be able to improve your products

Big data is a key that’ll explain specifically what your consumers really think about your product. Big data is even being used in medical research for companies that do personalized medicine or companion diagnostics, and need to analyze large amounts of biological data. You’ll be able to use your insight to easily get a better picture of your customers based out of different geographic areas and belonging to different demographic groups. Once you’ve seen how the product is perceived, you can easily raise efficiencies in key areas along the product process.
These are extremely valuable insights to present to your retailer buyer – enabling your company to rise above the noise from your competition.


Source:  10 Reasons Why Now Is the Time to Get into Big Data