The reality is, that some companies just don’t have a big enough presence in Walmart to warrant any kind of real IT spend. It was a huge deal when they brought on their first full time analyst that has some experience, and is located in Bentonville so that they can meet with their buyers whenever they need to. You get your company issued laptop, with a fresh install of Microsoft Office, a retaillink ID, and you are off and running, right?
No Problem – you are an excel wizard I bet. But as you are constantly maintaining horrific formula pages and trying not to break everything when your boss is requesting new feature engineering in their 20 page workbook, have you ever wondered, there has to be a better way?
You get what real data integration brings. Process automation that can refresh data without inventing the dashboard each and every single week. Feature engineering without such massive headaches. Adding new data sets to a model in a database, not a new tab. Oh the things you could do if you could get that time back.
Something that came about in MicroStrategy 10 was a new feature called Data Wrangling. This allowed an analyst to combine corporate data from the data warehouse with adhoc data and gave you come cleansing and harmonizing functionality. You’ve seen this need before. You get a data set where some states are all spelled out, while others have the two letter code. You see two rows for Texas (Texas and TX), and you have to scrub this in excel before it is usable. Or your data warehouse team does this in their ETL. Well, now you can do this directly in MicroStrategy with data you import yourself. You only need the $600 copy of MicroStrategy Desktop. Build a recipe and save it, and now this is semi-automated when you need to refresh your data. All you need to do is kick it off, and all of your cleansing tasks have been saved and are rerun.
I’ve always thought this was cool, ever since they showed this off at MicroStrategy world last year. But back to your budget of $0. Even $600 is going to be tough to get permission for. Well, I took a tour through a brand new product today that looks like it can do many of the same data wrangling techniques as MicroStrategy Desktop, but the cost is FREE. $0. Nada. It’s called Talend Data Preparation. It has a free version and a subscription version if you would like to work in a big environment and integrate with their other ETL platforms, but the free version looks like it is perfect for the lone rangers out there munging spreadsheets and cleaning data each week just so they can get to their analysis.
Check it out here: https://www.talend.com/products/data-preparation
You still need to kick it off when it’s time to refresh your data, but it lets you build recipes and save them, automating much of your work to refresh your data. It looks like it can connect to HTTP, local, and HDFS data sets.