Oct 1, 2023

Data stream has started and small steps taken | Readings

Web Dev

Progress: 5%

I’ve coded little TS this week aside from the Data Stream project. It has been a lot of terraform and deployments. However I’ve been tackling daily issues that arrive and I’ve been successfull to some extenct. So, I’ll consider it a win!

Code challenges

Progress: 4%

For this week I didn’t get any challenge done. I’m hoping that next week is more slow so that I can do not one but 2 challenges. Oh, and finnally decide if the priority is rust or Java on this section.

Data Stream

Progress: 10%

In the last week I’ve written a quick prototype to retrieve the data from explore.porto.pt and write it to a sqlite. The API is great and after doing some initial tests on a single stop I’ve easily started extracting data for the entire city. I got an issue where the response must return in less than 10 seconds but I’ve been able to solve it by removing unnecessary data fields. In terms of data analysis I’ve already gotten some interesting info but it clearly needs to be presented on a clean dashboard.

My next steps are:

Deploy the deno script
Setup frequent backups or use litefs
Do visual data analysis. It’s interesting to setup a realtime dashboard with sqlite and another for data analysis using duckdb

Communication

Progress: 0%

No progress on this front sadly. I’ve wanted to pick on a data presentation at Fidel but I’ve found myself doing a lot of tasks. I’ll be hoping to show a bit on wha we can do with duckdb for debugging use cases on next friday though.

Wellness

Progress: 0%

Since I came back from vacations I’ve had a a bit of a cold. I’m hoping to restart my training from Monday on. So, the plan is

1 soccer game
2 swimming sessions (going to try a new pool)
Daily warm-ups

Readings of the week

Somehow I’ve found myself adding more and more articles to omnivore. After all the reading I’ve gotten myself with a to-read list of 78 articles. With a quick check to the wordCount I feel I might need a whole nont-stop day to clean the list. So I might need to pruning 😅

However I’ve started writting more and more notes which I think is great! To complement this I think it might be interesting to get the notes for each article so that I can review my notes and write some of them on this status update.