Designing Data-Intensive Applications

I chose this book among others because I wanted to get out of reading about programming, methodologies, and other related stuff. I wanted to read something more abstract, not necessarily applicable to my immediate work, and boy did I get what I bargained for.

Well, let me start with complaints. First and foremost, the book is a bit long. I feel like it could have been compressed a bit. No, there isn’t much fluff, just several chapters could have been reorganized. Second, and no less subjective, the book at some point turns into a “horror movie,” where each passing chapter, things get more dangerous, complicated, and helpless. But I have to admit, there is a certain charm to such a delivery. Last grumble of mine: few examples of highly scaled and/or distributed systems. I would love to see more examples and discussions about them. Now I know the grumble isn’t entirely fair, since I got what I wanted – “more abstract” – but by the end of the book, I was left wanting more. I guess appetite comes during a meal.

Now let’s flip the page and go in the opposite direction: the examples in the book are quite interesting and educational. I loved the discussion about Twitter’s design and a couple more along the way. I personally don’t work with databases or other data-related systems that much, so the author fascinated me quite a bit. Some discussions are quite lengthy and detailed, mostly due to the complexity of the subject, but that’s what makes them so good, opening up your mind to different ways of thinking about programs, structures, and networks. The “horror movie” comes and goes, it makes you doubt everything you took for granted, even a CPU’s ability to add two numbers together. The author seems to project a lot of paranoia, the levels are similar to the amount of paranoia exhibited by system administrators. It sure gives a lot of perspective to developers, since lots of things are assumed to work properly and not to fail at the most opportune moment. One last thing: as I was reading the book, it reminded me of a bit of fun I had reading research papers at university. Some of the design patterns did emerge from academia, and perhaps it is a good direction if you have an interest in it.

I believe the title speaks for itself. If you want to design data-intensive applications, the book might be a good start. Perhaps it will give you a good insight and/or wider perspective. But if you are looking for hands-on material, the book might disappoint you. It is hard to tell whether the book is worth it or not. I had fun reading it; I believe it is well written, and I can see that the author put a lot of work into it. So I’ll leave you with this: if you are curious about the subject, go for it!

In a nutshell:
+: Well written
+: Broadens horizons
+: Lots of discussions and examples
+: No requirement for specialized knowledge
-/+: In a way, a long book; in a way, it needs to be longer
=: If you have any interest in application design, the book is an awesome way to start your journey.

Title: Designing Data-Intensive Applications
Author: Martin Kleppmann
Cover:

Goodbye 2022

Just a blink of an eye and 2022 is gone. In the past few years, I was quite optimistic about the upcoming years, but this year finally broke my back… Covid, inflation, and finally war. I hate to sound like an old fart, but it’s getting harder to look optimistically into the future. I wonder if I’m just getting grumpier or if the world is making less and less sense – perhaps both are the same side of the same coin.

In any case, I’m happy that 2022 is finally over. I’m looking forward to 2023 and the victory of Ukraine. Perhaps after the war is over, the world will slow down just a bit and become a little boring for a while, giving us all time to catch our breath. I hope the economy will survive without going into a deep recession and that we can all just get a break. Well, at least I can dream and wish.

Pragmatic Programmer

I been developing software for a while now and managed to read few books on the subject. Some books are general purpose, some with narrow focus on a particular subject in the development process. Pragmatic programmer is general purpose book on the subject.

I picked up the book in hopes of learning something new, something I haven’t seen or read before. Surprising enough I did find few things, but not nearly enough to justify going through the entire book. Now let me be very clear, the book itself is a good book, if you are starting out and want to improve your skills and understand what’s out there. However it might be a waste of time if you been in the game for a while and read a few things here and there. Also I can’t help but notice that some topics are not well covered even at a basic level (IMHO).

Since negatives are largely based on the amount of knowledge/experience a developer has, let’s talk about positives. The book is well written and relatively short. There is no fluff or metaphysical discussions, just practical and pragmatic advice. The book outlines and talks about all the useful basics that each developer should have, like: structures, clean code, testing, design, refactor, thought processes, personal & team behaviour, project organization and development methodologies. There is no deep dive into any subject, just essentials – which should spark enough interest in a developer to start researching more on any particular subject of interest. The author’s personal experience also comes in handy, some things don’t change over decades.

Overall, the book is good for inexperienced developers and for the experienced developers this book might be a trip down memory lane.

In a nutshell:
+: Well written & short
+: All the basics
+: Discussions
+: Some hands-on examples
-: Some subjects are not well covered even at basic level
=: Good book for new developers, but not much value for experienced devs.

Title: The Pragmatic Programmer, 20th Anniversary Edition your journey to mastery
Authors: Andy Hunt, David Thomas
Cover:

The Pragmatic Programmer, 20th Anniversary Edition: your journey to ...

Ukraine is fighting for 7 months now

Couple of days ago, putler announced “partial mobilization” of 300 thousand men. In reality it is a full mobilization and numbers looking towards a million. It is not a good news for Ukraine, but let’s not forget, at the beginning of the war, situation was much worse.

As I was thinking about the mobilization, I decided to find a trailer for a documentary and it brought back memories of how fearlessly and courageously Ukrainians fought and continuing to fight. Mobilization will not help putler, it will not save anyone or anything, just one more stop on the way to hell.

5 topics for yearly knowledge refresh

Recently one more senior developer decided to leave my team and the company. The event got me a bit sad, not only the team is loosing a good developer but it also means new developer will be joining the team and that means teaching the developer all the ropes.

I have been through this few times now and its really starting to get to me. It takes time for a developer to learn how to write clean code, test drive, refactor, not to mention learn all the ins and outs of the company’s systems.

In addition I keep noticing that a developer can take expensive courses, lets say on TDD and still lag behind – missing tests or writing too many. That got me thinking, is it possible to improve the situation by creating yearly refresh courses? Effectively new developers get to know all the essentials of development at the company and present developers get to fresh up on the existing practices and perhaps come up with improvements (or trash something that no longer brings value).

So here are 5 topics:

  1. Clean code
    It is important to learn and practice writing clean, easy to read and follow code. Clean code is a foundational knowledge, it effect all other practices in very fundamental way (from production to test code).
  2. Unit testing and TDD
    Testing is the prerequisite for continues delivery. Every developer must understand the value of testing and how it enables continues delivery. TDD is the best developer technique for writing valuable tests with the maximum reasonable coverage. Tests is code, it requires maintenance, tests must bring value and TDD is well established technique for doing so.
  3. Refactoring
    No one ever designs or writes perfect code. Moreover no one has crystal ball that predicts future business needs. Refactoring is important skill for continuously changing, adopting and improving code and system design.
  4. Higher order testing
    Beyond a system boundary, there are more systems. Developer must understand techniques and tools that are available for Integration, Contract and E2E Testing. Pros and cons must be weighted carefully in order to provide meaningful automated testing and short lead times. 
  5. Pipeline and environment
    Software systems no longer built locally and run on bare metal. Pipeline builds systems and those systems run in virtual environments. While developers are not DevOps (and probably will never be), it is important for developers to know how pipelines are developed, employed and maintained. How systems are packaged and run in docker under Kubernetes.

No Thanks

Over last nearly three months we’ve been hosting refugees. Perhaps it is a noble cause, may be admirable but we did it because it seemed as the right thing to do – people needed help and I and my wife were in a good position to help. I believe it is the biggest selfless act we performed to date. But I’m not writing this to brag, I’m writing this because I’m deeply sad and somewhat mentally struggling to process the past three months.

Let me start by stating somewhat fascinating (at least to me) fact; in the last three months, I learned nearly nothing about these people. All the time it felt like they didn’t want to be bothered. After initial few of attempts by me and my wife, we frankly gave up. It felt like they were ignoring us as much as possible. Most of the time we walked into a common area, they would leave (even though we encouraged the use of common area). Efforts to socialize were made but ultimately failed. We still talked but mostly when they needed something: taking to places, food, documents, resume and stuff like that. Granted, our family is having hard time with traditional breakfast/lunch/dinner – we simply don’t have “get together and eat”. But once in a while we do, at one point we got together for Chinese food dinner and invited the refugees. Kids (age 8 & 12) didn’t bother joining and their mom joined but the conversation didn’t really take off, as soon as food was done she left to her room and closed the door.

The kids are fascinating, simply because I never encountered such behaviour before. They never said “good morning”, unless I said “good morning” to them first. Forget about “good night”, may be “thank you” couple of times. I never met such a shy/private kids before. I remember meeting a super geeky boy a while back, but even though he was super shy, he still seemed pretty happy to be noticed and talked to. In this case kids seemed to regard me as necessary evil (at least that’s the way I felt), unhappy about any conversation attempts and never wanting anything. I never met any less curious kids in my entire life. Whenever I attempted to offer anything, they would immediately say “no”, in some cases even before I managed to finish a sentence. I have a six year old and it seems she received exactly the same treatment after initial several days of hanging out with the refugees. I know older kids don’t always like to hanging out with younger ones. But total ignore? It was painful and somewhat fascinating to watch my kid, she seemed to figure it out on her own and after sometime didn’t even notice the refugees.

I don’t know if we offended them in any way, I keep thinking about it but it’s not like we had extended conversations or discussions about anything. Most of the time they just stayed in their room, the door closed and didn’t communicate with us. We helped as much as we could – as much as I wish my family had received in a similar situation. We bought their airplane tickets, beds, food, clothing (some used, some brand new), helped with paperwork, provided a car to practice driving, pickup from the airport (which is 4 hours away), introduced to some people we knew and my mom somehow managed to get summer camp for the kids free of charge (typically about $300 per week per kid) – thank you city hall! Yet as time rolled on, we all got a feeling that we were bothering them, my wife mentioned that we were given the “cold shoulder”.

After nearly three months, one day, as I was making a tea in the kitchen, the refugee lady came out of the room and told me that she needs help – a ride to Brampton. I wasn’t sure if I wanted to spend my weekend driving refugees 4 hours away and then returning home. Despite advice of everyone (who knew what’s happening), I decided to drive the refugees to Brampton, to make sure they got from my place to their destination safely. As the day of departure approached, I was tensing up, I thought to myself: surely, the refugees will at the very least come out the room and say “thank you” and “goodbye”. I didn’t expect a dinner or tea or cookies, I mean after spending so much time avoiding us, I really didn’t expect much, no hugs, just a simple old “thank you”. My wife chuckled at me and gave me a new name: “Eugenius Simpletonius Maximus”, implying that I’m kindly naive. My mom simply said: “you will receive no gratitude”. Friend of mine said: “she will not thank you” and added that I’m hopeless optimist, yet, I still believed in basic human gratitude. The final evening came and went, my mom received no thanks even though both of them were home all day long. Me and my wife came home in the evening, the door remained shut with light inside – no thanks or goodbye came to us either. I got deeply upset.

Early morning the refugee lady asked for help – to take luggage to the car. So I did, I had a nut in my stomach, I briefly considered driving them to the nearest train station and leaving them there, but decided against it. So we drove for 4 hours towards destination in complete and utter silence. I still could not believe what had happened and I definitely didn’t know what to say, I was in complete and utter shock, the lady made no attempt to talk to me or even look at me the whole drive. The final moments were memorable, as we arrived, I unloaded the car, looked at the lady – she was busy with her kids, I walked to the car, looked at her again, she was still not looking at me, I sat into the car, still no attention to me, I started the engine – she didn’t even turn around and I drove off. I didn’t wait around, figuring everyone else was right at the end of the day and I was dead wrong.

45 minutes after I drove off, the lady decided to send “thank you” emails to my mom and to my wife, an email, after nearly three month of stay and all the help, we got an email! At this point I didn’t argue with my wife when she said “she is not done with us, she needs something from us” and she was dead right once again.

P.S: I’m still deeply upset, I know it will pass… but I’m still having a hard time comprehending the last three months.