For this week, I’ve spent it on reading. Below I got a list of 45 articles of highlighted videos and articles (basically cleaned the omnivore saves of last year). My main topics are AWS, Python, Spark as well as a bit of AI as this is the main focus of my work. The subjects are quite motivating for me although I have the constant feeling that the Web folk have it much better. The podcasts are great, the documentation is top notch and the quick iteration of tools makes for a healthy competition that in the ends helps the developers.

Readings of the week

• “An Introduction to PySpark” - Alex Ware (PyCon AU 2023) by PyCon AU
• Have the Boomers Pinched Their Children’s Futures? - with Lord David Willetts by The Royal Institution
• Should I Open Source my Company?
• AWS re:Invent 2023 - What’s new in AWS Lake Formation (ANT303) by AWS Events
• Slashing Data Transfer Costs in AWS by 99% · Bits and Cloud by Daniel Kleinstein
• Netflix Creates Incremental Processing Solution Using Maestro and Apache Iceberg by Rafal Gancarz
• “Rust for the Python dev” - Owen Lamont (PyCon AU 2023) by PyCon AU
• Learning FastAPI on hard-mode: getting out lots of the hardest (aw yeah, geo) data fast by PyCon AU
• Talks - Dan Craig: Testing Spacecraft with Pytest by PyCon US
• Talks - Mark Shannon: How we are making CPython faster. Past, present and future. by PyCon US
• “What can’t WebAssembly do?” - Katie Bell (PyCon AU 2023) by PyCon AU
• Announcing: DuckDB code snippet sets with MotherDuck Sharing by Ryan Boyd
• Zendesk Moves from DynamoDB to MySQL and S3 to Save over 80% in Costs by Rafal Gancarz
• Database Isolation Is Broken and You Should Care by Chris Riccomini
• Trying chDB, an embeddable ClickHouse engine by Anton Zhiyanov
• eBPF: Unlocking the Kernel [OFFICIAL DOCUMENTARY] by Speakeasy Productions
• New AWS Lambda scaling controls for Kinesis and DynamoDB event sources | AWS Compute Blog
• The Query Strikes Again - Slack Engineering by Eduardo Ortega
• ELTP: Extending ELT for Modern AI and Analytics | Airbyte by AJ Steers
• 6 Habits Of High-Performing Teams by SeattleDataGuy
• Pursuit of wicked smartness in VS Code by Microsoft
• Felix’ Blog - GitHub Actions are a Problem by Felix Knorr
• MotherDuck: Making PySpark Code Faster with DuckDB by MotherDuck
• git rebase: what can go wrong? by Julia Evans
• Spark, Dask, DuckDB, Polars: TPC-H Benchmarks at Scale by Coiled
• Talks - Moshe Zadka: pyproject.toml, packaging, and you by PyCon US
• Understanding Parquet, Iceberg and Data Lakehouses at Broad by David Gomes
• Zendesk Moves from DynamoDB to MySQL and S3 to Save over 80% in Costs by Rafal Gancarz
• Database Isolation Is Broken and You Should Care by Chris Riccomini
• Introducing Fly Kubernetes · The Fly Blog
• Crush Your Interviews with the Power of Storytelling by Dave Anderson
• New AWS Lambda scaling controls for Kinesis and DynamoDB event sources | AWS Compute Blog
• Distributed Transactions at Scale in Amazon DynamoDB by Akshat Vig
• Portable EPUBs by Will Crichton
• Do we think of git commits as diffs, snapshots, and/or histories? by Julia Evans
• “Human error” means they don’t understand how the system worked – Surfing Complexity by Lorin Hochstein
• SQLite Forum: JSONB has landed
• It’s About Time! - Marc’s Blog by Marc Brooker
• git branches: intuition & reality by Julia Evans
• Incremental Processing using Netflix Maestro and Apache Iceberg | by Netflix Technology Blog | Nov, 2023 | Netflix TechBlog by Netflix Technology Blog
• Psyberg: Automated end to end catch up | by Netflix Technology Blog | Nov, 2023 | Netflix TechBlog by Netflix Technology Blog
• Diving Deeper into Psyberg: Stateless vs Stateful Data Processing | by Netflix Technology Blog | Nov, 2023 | Netflix TechBlog by Netflix Technology Blog
• Streamlining Membership Data Engineering at Netflix with Psyberg | by Netflix Technology Blog | Nov, 2023 | Netflix TechBlog | Netflix TechBlog by Netflix Technology Blog
• How git cherry-pick and revert use 3-way merge by Julia Evans
• Summing columns in remote Parquet files using DuckDB | Simon Willison’s TILs

Goals for this week

I’m making my only goal for this week to advance on Advent of code. With a goal of at least a challenge per day I’ll be quite satisfied.

• Advance advent of Code to at least day 20