How we run Gemini at scale across billions of posts
Modash utilizes large language models (LLMs) to process billions of social media posts efficiently. The company has developed production pipelines that enhance data quality by understanding content rather than relying on traditional pattern-matching methods. This approach helps reduce false positives and improves the overall value of the data provided to customers.
- ▪Modash's dataset grows by millions of posts daily, necessitating the use of LLMs for data processing.
- ▪The company employs a multi-cloud setup to manage operational complexity and costs associated with LLM inference.
- ▪LLMs are used to extract structured meaning from messy, multilingual social content, improving data quality.
Opening excerpt (first ~120 words) tap to expand
Using LLMs with billions of inputs in a multi-cloud setupAt Modash we sit on top of a creator-discovery dataset that grows by millions of posts every day. A growing slice of that pipeline now runs through LLMs.This massive volume of inference adds up on our cloud bills and our operational complexity. In this article you will learn how we actually run an LLM against billions of inputs without going broke.Why We Use LLMsIs the AI hype worth it? Do LLMs have any real use beyond being a 24/7 chatbot? We think so, and over the last year we’ve shipped several production pipelines where LLMs are visibly improving the data we deliver to our customers.Several of those pipelines exist to extract structured meaning from messy, multilingual, multimodal social content.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Modash.
Discussion
0 commentsMore from Modash
