notes-design.md

Ashwin Jayaprakash CC-BY-SA-4.0

Table of Contents

• About
• Problem solving
• Programming
• Computer fundamentals
• IDEs
• Best practices
• Code Katas
• Code reviews
• System design basics
• Performance and deep dives
• Concurrency
• Distributed systems
• SQL
• Cloud
• Testing
• Documentation and information management
• Engineering excellence
• Operational excellence
• Project maintenance and contributions
• Learning and leadership
• "Awesome" lists

Created by gh-md-toc

About

This is an opinionated collection of reading material that I've collected over the years to help me become a better Engineer. It is not an exhaustive list. I share some more on a somewhat regular basis on my Tech Blog. I also do a lot of general reading on leadership, culture, org building etc but I haven't found the time to share those.

Problem solving

• Situation, task, action, result
• The Five W's (and H) approach

Programming

• How To Write Unmaintainable Code
• Clean Code by Robert Cecil Martin (Cheat sheet)
• Principles of Effective Developers

Computer fundamentals

• Writing an OS in Rust
• Writing a file system from scratch in Rust
• What's in a Linux executable?
• Wizard zines

IDEs

• Learn to use the IDE/IntelliJ well
• IntelliJ IDEA Tips and Tricks by Anton Arhipov
• IntelliJ Super Productivity in 45 Minutes
• IntelliJ IDEA. Debugger Essentials

Best practices

• General
  • SOLID design patterns
  • Summary of OCP, SRP etc
  • Venkat Subramaniam Design Patterns talks
  • GRASP design patterns
  • Domain-driven design by Eric J. Evans
  • Applying UML and patterns by Craig Larman
  • Enterprise Integration Patterns by Gregor Hohpe
  • Refactoring, patterns, anti-patterns
  • Package by Feature
  • Microservice architecture
  • Laws of UX
  • Hexagonal Architecture and Payments Monolith decomposition - Johnny Willer
  • Mateusz Chrzonstowski - My understanding of DDD & Clean Architecture on the example
  • I Made Everything Loosely Coupled. Does My App Fall Apart? - Gregor Hohpe
• Java
  • Best Practices for Designing and Implementing a Library in Java
  • Effective Java, Third Edition Keepin' it Effective by Joshua Bloch
  • Josh Bloch on API design
  • Google Best Practices for Java Libraries
  • Secure coding
  • Java Design Patterns
  • Gang of Four Patterns in a Functional Light
  • Revisiting Design Patterns after 20 by Edson Yanaga
  • Data-Oriented Programming in Java
  • Programming Principles - slightly different from the ones above
  • 10 Reasons Why we Love Some APIs and Why we Hate Some Others by Lukas Eder
  • Let's make a contract: the art of designing a Java API by Mario Fusco
  • Evolving Java-based APIs / Achieving API Binary Compatibility
  • Trustin Lee: Writing a Java library with better experience
  • Date-Time Design Principles
  • The art of building Java APIs: Do's and Don'ts - Jonathan Giles
  • JooQ API design
  • Java API design checklist
  • 12 recipes for using the Optional class as it’s meant to be used
  • Avoid checked exceptions
  • A corner case cheat sheet for Java and JVM languages
  • Principles of Fluent API Design by David Beaumont
• Style
  • Google style guide
  • A list of awesome style guides
  • Use tools to enforce a consistent, standard coding style (Checkstyle/PMD/FindBugs/SpotBugs/ErrorProne/Infer to do the first level checks)
  • Use IDEs and tools to format and fix compiler/IDE warnings
• APIs and CLIs
  • Microsoft API guidelines
  • Google API design
  • Kubernetes REST API conventions
  • RESTful APIs and Events, created by Zalando
  • REST API tutorial
  • Avoid Data Corruption in Your REST API with ETags
  • Boost Your REST API with HTTP Caching
  • REST API Checklist - Kenneth Lange
  • 7 Tips for Designing a Better REST API
  • A guide to help you write better command-line programs

Code Katas

• c-guntur/paneer-tikka-masala: A Code Kata using a cooking recipe
• c-guntur/java-katas: One repo to rule them all. All Java Katas in one repo
• eclipse/eclipse-collections-kata: Collections Kata
• vmzakharov/mutate-test-kata: Code kata: using mutation testing to improve quality of unit tests
• forax/design-pattern-reloaded: Implementation of GoF design patterns in Java 8
• forax/java-guide: A guide of modern Java (Java 17)
• BNYMellon/CodeKatas: BNY Mellon Code Katas

Code reviews

• Code review matters and manners with video
• Building a code review culture
• Google Code Review Developer Guide
• Chromium Project - Respectful Code Reviews
• Chromium Project - Respectful Changes
• A Software Engineer’s Guide to Giving Feedback that Actually Matters
• The Code Review Pyramid

System design basics

(Also see 12 Factor, FURPS / Functional Usability Reliability Performance Supportability)

• Requirements
  • Goals
  • Non goals
  • Scale
  • Constraints
  • Success measures
• Design - Basics
  • Customer journeys (Visit -> Search -> Add -> Sign in/up -> Checkout -> Order -> Invoice -> Payment -> Fulfillment -> Shipment -> Confirmation/Return/Cancel/Refund/Status/Notify)
  • APIs and Entities - CRUD
  • Services - Foreground and background
  • Stores
  • Reliable (consistent, available), scalable (latency, goodput), observable, maintainable, secure
• Design - Principles
  • Customer first
    • MVP
    • Necessary (support), sufficient (resistance), nice to have
  • Critical thinking
    • ROI
    • COGS
    • KPIs
    • Data and experiment driven
  • Tradeoffs
    • Simple, easy to reason and scale
    • Mechanical sympathy
    • Reduce toil and undifferentited heavy lifting
• Scope
  • CDN / Edge network / POP, Region / availability zone / cell
• System - Loadbalancer
  • Healthcheck
  • Routing
  • DNS updates
  • TLS and Certs
• System - API Gateway
  • Protocols
  • Firewall
  • Rate limiting
  • Bounds check
  • Authz and authn
  • Routing
  • Logging
• Concept - Resilience
  • Timeout
  • Retry
  • Jitter
  • Bulkhead
  • Circuit break
  • Backpressure
  • Memoize
  • Fault domain
• Concept - Reads, writes and operations
  • Space and compute complexity
    • Arrival time, wait time, service time
    • Read and write ratio
  • Consistency, availability, durability
    • Atomic, transactional, lease, lock
    • Eventually or strongly consistent
    • Durable
    • Synchronous, asynchronous
      • Commands (need to be worked upon, ideally by a single worker)
      • Events (already happened)
      • Query (deterministic queries that don't have random numbers etc produce repeatable results)
      • Checkpoint, write after/before logs, replay-idempotence, compensation-rollback
  • Critical path
    • Crosstalk, coordination, contention
    • Partition, replication, affinity, anti-affinity
    • Schema on read or write, filter and aggregate on read or write
    • Deduplication, pre-fetching, pre-aggregating
    • Batching, compression, retention, tiering
    • Online vs offline - Analytics, notifications, repairs, reconciliation
    • Caching, invalidation, summary or probabilistic structures
    • Scoring, sorting, constraint satisfaction
• Concept - Algorithms
  • Loadbalancing
    • Least used
    • Power of two random choices
    • Round robin
    • ...
  • Hashing
    • Consistent hash
    • Modulo hash
    • Rendezvous hash
    • ...
• Concept - Security
  • Authenticate and authorize
  • Encryption at rest and motion
  • Sign and verify
  • Governance, risk and compliance
  • Audit trail
  • Principles
    • SSO, RBAC ...
    • Least privilege
    • Short lived tokens
    • Key rotation
    • ...
• Concept - Operations
  • Measure, monitor, alert - Detect, escalate, respond, prevent
    • Cost
    • Performance
    • Errors
    • Capacity
    • Disruption and error budgets
    • Test - chaos and synthetic
    • SLO
  • Deploy safely
    • Feature flags and canary/blue green rollouts
    • Reduce toil
      • Auto repair
      • Auto scale up and back to zero

Performance and deep dives

• General
  • Latency numbers every programmer should know
  • Misery Metrics & Consequences - Gil Tene
  • Square Engineering's "Fail Fast, Retry Soon" Performance Optimization Technique
  • Optimizing Servers for High-Throughput and Low-Latency at Dropbox
• Java
  • JVM bytecode
  • JVM internals
  • No Free Lunch? Memory Allocation in the JVM
  • Java at Speed
  • Aleksey Shipilёv's posts describing some elementary piece of knowledge about JVM
  • JVM Anatomy 101
  • Time To Really Learn Generics: A Java 8 Perspective
  • Mastering the mechanics of Java method invocation
  • Understanding Java method invocation with invokedynamic
  • Behind the scenes: How do lambda expressions really work in Java?
  • Understanding Gradle
  • Secrets of Performance Tuning Java on Kubernetes by Bruno Borges

Concurrency

• Brian Goetz on concurrency
• Venkat Subramaniam concurrency talks
• Concurrent programming in Java by Doug Lea
• Roman Leventov's Code Review Checklist for Java Concurrency

Distributed systems

• General
  • An introduction to distributed systems
  • High Scalability
  • Not Just Scale - Marc Brooker
• Resilience
  • Stability Patterns and Antipatterns, Michael Nygard
  • 10 patterns for more resilient applications - Uwe Friedrichsen

SQL

• Practical SQL for Data Analysis - Haki Benita
• Awesome SQL Tips and Tricks - Voxxed Days Cluj - 2019
• How Query Engines Work
• How Modern SQL Databases Come up with Algorithms that You Would Have Never Dreamed Of by Lukas Eder - YouTube

Cloud

• Jerry Hargrove - Cloud Diagrams & Notes
• open-guides/og-aws: Amazon Web Services — a practical guide
• Amazon Web Services In Plain English
• Amazon Web Services (also in plain English)
• AWS services explained in one line each

Testing

• Test Desiderata
• Cassandra testing guidelines
• Don't be mocked by your Mocks: Listening to your Tests
• Testing legacy code when you dislike tests (and legacy code) by Maeve Revels
• Effective Unit Testing by Eliotte Rusty Harold
• Write awesome tests by Jeroen Mols
• An Ultimate Guide To BDD
• The lazy programmer's guide to writing thousands of tests - Scott Wlaschin
• Property based testing
• Integrating PIT Mutation Testing and GitHub Pages with GitHub Actions

Documentation and information management

• US Government plain language guidelines
• Technical Writing Courses for Engineers
• Effective software design documents & A generic design document template for documenting Micro services
• Google Developer Documentation Style Guide
• The Modular Documentation Project Source Repository
• The Art of Code Comments - Sarah Drasner
• How to Write a Git Commit Message
• Conventional Commits - specification for adding human and machine readable meaning to commit messages
• If Hemingway Wrote JavaDocs
• Keep a Changelog

Engineering excellence

• We invested 10% to pay back tech debt; Here's what happened

Operational excellence

• How to Best Use MTT* Metrics to Optimize Your Incident Response

Project maintenance and contributions

• Tanzu Community Engagement guidelines, community health measurements, and goals
• Cassandra contribution guide has a nice page on code review among other things
• eclipse collections contributor guide
• Strimzi contributor guide
• Contributing to Seldon Core

Learning and leadership

• Barbara Oakley: "Learning How to Learn"
• Reading Research: A Guide for Software Engineers
• Engineering You: Martin Thompson
• Understanding the role of a principal engineer

"Awesome" lists

• (My very own) Ashwin Jayaprakash's tech blog
• "Awesome" lists
• A curated list of awesome Java frameworks, libraries and software