The tech stack
Building FootballMeme has given me a chance to learn about a bunch of awesome cloud services, as well as knock the rust off my coding skills. Here's my tech stack:
- Vercel because all the hyperscalers are way too damn complicated and expensive. Vercel has a fantastic free tier for hobbyist projects.
- Supabase is my database. Not just because I work there. It's awesome.
- My stack is Next.js.
- I use the OpenAI API.
Queues and cron jobs
I have a Supabase cron job running every 30 minutes that scrapes all the major American football sites for new articles and content. When it finds a link, it puts it on a Supabase queue.
I have a second cron job that runs every 30 seconds and dequeues one link from the Supabase queue and determines if it is a new link. If it is, it examines the domain and puts the link in a different queue. I maintain a queue for each domain (profootballtalk.com, espn.com, nfl.com, YouTube, etc.)
Each of these domain-specific queues has its own corresponding cron job. Every 15 seconds it checks the queue for new links and if it finds one, it dequeues the link and processes it.
Upon processing the link, I use the OpenAI API to identify the following based on the content of the article:
- Players
- Coaches
- Owners
- Teams
I then store the content of the article, an embedding (using pgvector) of the article, as well as all the players, coaches, owners, and teams associated with the article.
My front-end app then queries for the latest articles and displays the headline and tags. I also have a search bar in the front-end app so you can filter by players, coaches, owners, and teams.
Trending topics
A feature I miss a ton now that I've ditched Twitter is being able to see who is currently trending in the NFL. It was always an easy way for me to get caught up on what was happening right now.
So, I wanted to build a trending topics feed into FootballMeme.
Whenever a player, team, coach, or owner is mentioned, I ascribe some points depending on the context. For example, news about an injury is one point, but news about winning an award is nine points. These points are determined by using OpenAI to analyze the article's contents.
On top of that, every time someone clicks on an article, I add a point to the players, teams, coaches, or owners mentioned in the article.
Together with the publication date of the article or post, this helps me determine which players and teams are currently trending.
Trending topics are stored in Supabase. Using Supabase Realtime, and each time the database is updated, the front-end updates with the latest trending topics.
About me
I'm Prashant. I am the author of Picks and Shovels: Marketing to Developers During the AI Gold Rush, the definitive guide to developer marketing, developer relations, and technical product marketing.
I'm also the Product Marketing Manager at Supabase.