Market Analysis At Scale

How to perform an automated market analysis.

Market Analysis At Scale
Guillaume Odier

Guillaume Odier



Thanks to the web, we have the amazing opportunity to create new business models and distribution channels. However, this process is hard to master and even harder to optimize.

In this case study, our corporate client sells machines and spare parts. It deals with a multitude of distributors, themselves using over 20 websites to redistribute spare parts.

Our job is to retrieve every spare parts references on each website used in the distribution process. This data feeds an internal argus that optimizes the selling of spare parts for each of its distributor. Indeed, each additional euro can lead to a significant increase in our client's margin.

In this case, we automate a market analysis at scale on a monthly basis.

The overall objective is to adjust the price of its spare parts, as well as gaining valuable insights on the US market.

Proof Of Concept (POC)

First things first, we have to demonstrate our expertise and the benefits such a solution could bring.

To begin the project, we start of with the development of only two bots out of the +20. We chose the most significant platforms in terms of data and technical challenges.


We take a look at the metrics related to the project, the possible adjustments and the successes & failures.

At the end of this phase, we establish a roadmap and a budget for the entire duration of the project.


Once the budget has been validated and adjusted, we assign a dedicated developer over a defined period. Indeed, this project requires to develop 20 bots. Thanks to the POC, everything is easy to setup regarding data integration since with now have a perfect understanding of the needs.

In total, +20 bots harvest monthly distributors' websites to power up the argus, resulting in million of additional revenues.

Lorem ipsum dolor sit amet, consectetur adipiscing elit.

 Captain Data, All rights reserved.