Ones and Zeros: Tech Shows Its Ugly Face
When the Cloud and Coding Throws Curve Balls, the Learning Curve Hits Again
A warm welcome to all new readers! I’m glad you are on board.
I’m Michael and on a mission to make beating the stock market as easy for you as choosing your favorite ice cream. I write an algorithm that does exactly that.
Read the full story here.
"In the middle of difficulty lies opportunity."
- Albert Einstein
They say necessity is the mother of invention, but let’s call it what it is—a beast of a week that slaps your face, snatches away sleep and pushes you out of your fuzzy comfort zone.
This is a story where code is king and every byte-sized decision has its price—at the same time, it’s like the satisfaction that comes after a rigorous workout.
Switching Databases: From Chill to Challenging
I kicked things off with what I thought would be a walk in the park: swapping out databases. SQLite was like the cool neighbor girl you love to hang out with - easygoing, flexible, and low-maintenance. But MySQL? That's where the simplicity ended. She's like the high-maintenance diva date who won't let you get a word in edgewise unless it's exactly what she wants to hear (with the typical diva hand move). To make any two pieces fit together, it's not just about similarity - they've got to be twins, down to the last detail. It's as if you're not just telling her you speak English; she wants to know if you can do the Queen's accent from 1995.
The Cloud Part 1: A Jargon-Heavy Jungle
Then I hit the cloud – where the land of possibilities stretches as far as the horizon, yet the path is covered by a fog of jargon and specifications. You get flooded with terms and choices that sound more like a foreign language than user options. I had to sift through all that lingo, trying to figure out what in the world I needed – I/O’s this, EC2 that, and so on. It took me a solid three days buried in documentation to even begin to start.
The Cloud Part 2: Speed vs. Spend
Here's the thing about the cloud - it's like ordering à la carte; every item adds to your tab. You're always weighing up speed against your wallet. Pour enough cash into it, and sure, it'll run like a dream. But we're not there yet. So, I put on my engineer's cap and got to work, looking for a sweet spot.
The answer?
Batching. Instead of letting each ticker trickle into the database one by one, I grouped them and sent them in as a team. It's faster because we're not shaking hands with the database every single time. And if my current setup puffs with the biggest batch and I’m not satisfied with its speed, then I'll think about spending more on a beefier setup - but not a second before.
The Cloud Part 3: Coding at Full Throttle
Speed's my thing, and I'm always chasing it, especially in my code. That's where Python and its cProfile come in, helping me sniff out the bottlenecks. Think of it like finding out I've been racing a sports car in neutral. Awesome, right? Lots of noise and nothing happens. cProfile pointed out I was restarting the engine with every download - a total facepalm moment. Once I figured that out, I kept the engine running, and boom - what used to take 17 hours now takes 3. Very satisfying.
Teamwork with the Creatives
I also teamed up with a designer with a great energy. We're cooking up an interface for my website tool that's not just functional, but a joy to use.
So with that, it's back to the drawing board.
Work's not gonna do itself!
Progress
May I repeat my question from my last article.
What would you like to see on the final report? Please leave a comment below so I can adjust the report to your wishes.
I could flood you with data of course, but I guess less is more since the aim of this service is to make your life easier, not harder.
Plan
Continue the cloud transition and do a first POC.
What’s on my head
Guess what … :-)
Smash that heart below and let's ride the learning curve together!
Have a great day StockStar!
Michael
Nuggets I’ve enjoyed:
A Note I’ve enjoyed: 20 Investing Mistakes
- - their subscription is worth every penny.
If you think somebody should read this, share it, and make them happy.
Disclaimer:
The information in this article is my personal opinion. I’m not a certified investment professional. It is not consulting, nor does it constitute investment recommendations.
I do my research carefully and follow my personal investment strategy.
The stock market is a complex building with its own rules. There are no rules set in stone, like the rules of physics.
Therefore, use the contents of this newsletter at your own risk and do your own research as well. Investing in the stock market can lead to a total loss of the capital invested.
Opportunity comes to the prepared mind the saying goes ... but they forgot to say how long it takes to be prepared haha ... joke aside, a lot of work feels invisible many times, until it does not ... .
On IT challenges, your post reminded me a meeting I did participate in where the big Board did participate, it was about unifying the IT/databases landscape (300+ systems across countries with different regulations etc), have rather one system, cleaned data, nice interface for sales/risk/audit all ... head of project comes in, presents how to fix and all ... but also said what budged is needed, time, how many man days etc ... and one board member that never worked really, he was a 'manager' already at 24 and board member at 40, only had high school (no clue how you become a Board member in a listed big company, well I now know how but different story) said and yelled: 'whattt, this many millions and time needed? but it's just one bloody IT interface to spit some figures and various views for risk/sales ...' ... that was another moment when I realised how disconnected from reality many executive 'leaders' are from reality ... many living in the nice and smooth 'cloud' ...
My job was just to analyse the whole spaghetti IT landscape and to inform the board that we spend 80% of people/time/money to MAINTAIN all the systems patched and working, only 20% for the new future digitalisation ... and they did not believe me ... :)