A lot of problems we solve at Squad deal with images. Images to be tagged, labeled, moderated, transcribed etc. And there is this thing about images, they are generally heavier than text, digitally. And if our contractors (users of our app) spend more money on internet packs/plans than what they earn, or if they have … Continue reading Leveraging AWS Lambda for Image Compression at scale
Hacks should be quick. And so should be the articles about them. Photo by Caspar Rubin on Unsplash Problem We needed to calculate medians, percentiles for some quantities for our own ETL system (note to self: write a post on this) grouped by certain fields. Some options we had: Extra: But Django says this will … Continue reading Hack Django ORM to calculate Median and Percentiles (Or make annotations great again!)
Tl;dr: Avoid joins on large tables and evaluate parts of queries beforehand to get 100–10,000x performance gains! As mentioned in a previous post, because of some of our tables growing in size, our queries started performing poorly which resulted in a performance hit to our most used APIs. It was time we revisit some of … Continue reading Blazingly fast querying on huge tables by avoiding joins
Short story of how we reduced the response time of some of our admin pages by 1000x. The Problem One of our tables grew to over 25 million rows. Which, while not a large number, makes the most frequent queries slow. A direct effect it had was on our Admin panel. The change_list form for … Continue reading Estimated counts for faster Django Admin change_list
What do you do when you are asked to create a notification system where the objects to be notified get added/removed at runtime? You stop giving a shit about them, and code for the interface.
What do you do when the OO concepts you have abided by since your school days land you in trouble? You relearn them.