Skip to content

Commit 13330f9

Browse files
wendev27wendev27
committed
docs(checkpoint): complete checkpoint 22 and practices - databases and caching foundations
1 parent 82d950e commit 13330f9

13 files changed

Lines changed: 793 additions & 3 deletions

‎README.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -92,10 +92,13 @@ This repository was created to:
9292

9393
## 🗄️ Phase 6 — Databases & Caching
9494

95-
- [ ] PostgreSQL
95+
- [x] PostgreSQL in Docker
96+
- [x] SQL CRUD
97+
- [x] Docker Volumes
98+
- [x] Persistence
99+
- [x] Database Migrations (conceptually)
100+
- [x] Backups
96101
- [ ] Redis
97-
- [ ] Database Backups
98-
- [ ] Database Migrations
99102

100103
---
101104

‎checkpoints/checkpoint-22-databases-redis-and-caching-foundations.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# Checkpoint 22 — Databases & Caching Foundations
2+
3+
## 🎯 Objective
4+
5+
Understand how modern applications store, persist, modify, back up, and cache data using PostgreSQL and Redis inside Docker containers.
6+
7+
Learn the differences between relational databases and in-memory caching systems while building a strong mental model of how production systems manage data.
8+
9+
---
10+
11+
## 📚 Topics Covered
12+
13+
- PostgreSQL in Docker
14+
- Docker volumes and persistence
15+
- CRUD operations in PostgreSQL
16+
- SQL fundamentals
17+
- Database schema design
18+
- Database migrations (conceptually)
19+
- Database backups and recovery
20+
- Redis fundamentals
21+
- Redis data structures
22+
- Redis caching
23+
- Time To Live (TTL)
24+
25+
---
26+
27+
## 🤔 Questions Explored
28+
29+
- Why use PostgreSQL inside Docker?
30+
- What problem do Docker volumes solve?
31+
- Why does data survive container deletion?
32+
- What is the difference between schema and data?
33+
- How do database migrations work?
34+
- How does Laravel's `php artisan migrate` work internally?
35+
- Why are backups necessary even when using volumes?
36+
- Why is Redis significantly faster than PostgreSQL?
37+
- How does caching reduce database load?
38+
- How do applications implement sessions, OTPs, and expiration systems?
39+
40+
---
41+
42+
## 🔬 Labs and Experiments
43+
44+
### PostgreSQL Lab
45+
46+
- Ran PostgreSQL inside Docker.
47+
- Resolved a port conflict with the local PostgreSQL installation.
48+
- Connected to PostgreSQL using `psql`.
49+
- Created tables manually.
50+
- Performed CRUD operations.
51+
- Explored Docker volumes and persistence.
52+
53+
### Database Migrations
54+
55+
- Created migration files:
56+
57+
```text
58+
migrations/
59+
├── 001_create_users.sql
60+
├── 002_create_tasks.sql
61+
└── 003_add_phone_to_users.sql
62+
```
63+
64+
- Simulated Laravel migrations manually.
65+
- Applied migrations in sequence.
66+
- Explored schema evolution.
67+
68+
### Database Backups
69+
70+
- Exported the database using:
71+
72+
```bash
73+
docker exec postgres-lab pg_dump -U admin devsecops_lab > backup.sql
74+
```
75+
76+
- Examined the generated SQL backup.
77+
- Simulated accidental data loss.
78+
- Discussed backup security and encryption.
79+
80+
### Redis Lab
81+
82+
- Started Redis in Docker.
83+
- Connected using `redis-cli`.
84+
- Practiced Redis commands:
85+
86+
```redis
87+
SET
88+
GET
89+
INCR
90+
TTL
91+
EXPIRE
92+
LPUSH
93+
LRANGE
94+
SADD
95+
SMEMBERS
96+
HSET
97+
HGETALL
98+
KEYS
99+
```
100+
101+
- Built a mental model for caching.
102+
103+
---
104+
105+
## 🧠 Key Concepts
106+
107+
### PostgreSQL
108+
109+
```text
110+
Persistent storage
111+
Disk-based
112+
Relational
113+
Source of truth
114+
```
115+
116+
### Redis
117+
118+
```text
119+
In-memory
120+
Extremely fast
121+
Key-value store
122+
Temporary storage
123+
Cache layer
124+
```
125+
126+
### Cache Flow
127+
128+
```text
129+
User
130+
↓
131+
Backend
132+
↓
133+
Redis
134+
↓
135+
PostgreSQL
136+
```
137+
138+
### Docker Volume Persistence
139+
140+
```text
141+
Container ❌
142+
Volume ✅
143+
```
144+
145+
### Common Redis Use Cases
146+
147+
- Sessions
148+
- OTP codes
149+
- Rate limiting
150+
- Notifications
151+
- Caching
152+
- Password reset links
153+
- Queues
154+
155+
---
156+
157+
## 💭 Reflection
158+
159+
This checkpoint revealed that many features provided by frameworks such as Laravel are abstractions built on top of databases, caching systems, and infrastructure.
160+
161+
Concepts that once felt magical—such as migrations, sessions, expiration timers, and caching—became easier to understand after exploring PostgreSQL and Redis directly.
162+
163+
A major realization was that Redis does not replace PostgreSQL. Instead, Redis reduces unnecessary work by storing frequently accessed data in memory and allowing applications to avoid repeating expensive database operations.
164+
165+
---
166+
167+
## 🚀 Next Checkpoint
168+
169+
- Monitoring and Observability
170+
- Logs
171+
- Metrics
172+
- Health checks
173+
- Prometheus
174+
- Grafana
175+
- Dashboards

‎labs/databases/postgresql-lab/backup.sql‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
--
2+
-- PostgreSQL database dump
3+
--
4+
5+
\restrict YWhz4ALTeDQK2TekRCVweACNkZvAWoY2Ni1yU5HfULcpAYfR8ccbGgFwSoWW7we
6+
7+
-- Dumped from database version 17.10 (Debian 17.10-1.pgdg13+1)
8+
-- Dumped by pg_dump version 17.10 (Debian 17.10-1.pgdg13+1)
9+
10+
SET statement_timeout = 0;
11+
SET lock_timeout = 0;
12+
SET idle_in_transaction_session_timeout = 0;
13+
SET transaction_timeout = 0;
14+
SET client_encoding = 'UTF8';
15+
SET standard_conforming_strings = on;
16+
SELECT pg_catalog.set_config('search_path', '', false);
17+
SET check_function_bodies = false;
18+
SET xmloption = content;
19+
SET client_min_messages = warning;
20+
SET row_security = off;
21+
22+
SET default_tablespace = '';
23+
24+
SET default_table_access_method = heap;
25+
26+
--
27+
-- Name: tasks; Type: TABLE; Schema: public; Owner: admin
28+
--
29+
30+
CREATE TABLE public.tasks (
31+
id integer NOT NULL,
32+
title text NOT NULL,
33+
completed boolean DEFAULT false,
34+
created_at timestamp without time zone DEFAULT CURRENT_TIMESTAMP
35+
);
36+
37+
38+
ALTER TABLE public.tasks OWNER TO admin;
39+
40+
--
41+
-- Name: tasks_id_seq; Type: SEQUENCE; Schema: public; Owner: admin
42+
--
43+
44+
CREATE SEQUENCE public.tasks_id_seq
45+
AS integer
46+
START WITH 1
47+
INCREMENT BY 1
48+
NO MINVALUE
49+
NO MAXVALUE
50+
CACHE 1;
51+
52+
53+
ALTER SEQUENCE public.tasks_id_seq OWNER TO admin;
54+
55+
--
56+
-- Name: tasks_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: admin
57+
--
58+
59+
ALTER SEQUENCE public.tasks_id_seq OWNED BY public.tasks.id;
60+
61+
62+
--
63+
-- Name: users; Type: TABLE; Schema: public; Owner: admin
64+
--
65+
66+
CREATE TABLE public.users (
67+
id integer NOT NULL,
68+
username text NOT NULL,
69+
email text NOT NULL,
70+
created_at timestamp without time zone DEFAULT CURRENT_TIMESTAMP,
71+
phone text
72+
);
73+
74+
75+
ALTER TABLE public.users OWNER TO admin;
76+
77+
--
78+
-- Name: users_id_seq; Type: SEQUENCE; Schema: public; Owner: admin
79+
--
80+
81+
CREATE SEQUENCE public.users_id_seq
82+
AS integer
83+
START WITH 1
84+
INCREMENT BY 1
85+
NO MINVALUE
86+
NO MAXVALUE
87+
CACHE 1;
88+
89+
90+
ALTER SEQUENCE public.users_id_seq OWNER TO admin;
91+
92+
--
93+
-- Name: users_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: admin
94+
--
95+
96+
ALTER SEQUENCE public.users_id_seq OWNED BY public.users.id;
97+
98+
99+
--
100+
-- Name: tasks id; Type: DEFAULT; Schema: public; Owner: admin
101+
--
102+
103+
ALTER TABLE ONLY public.tasks ALTER COLUMN id SET DEFAULT nextval('public.tasks_id_seq'::regclass);
104+
105+
106+
--
107+
-- Name: users id; Type: DEFAULT; Schema: public; Owner: admin
108+
--
109+
110+
ALTER TABLE ONLY public.users ALTER COLUMN id SET DEFAULT nextval('public.users_id_seq'::regclass);
111+
112+
113+
--
114+
-- Data for Name: tasks; Type: TABLE DATA; Schema: public; Owner: admin
115+
--
116+
117+
COPY public.tasks (id, title, completed, created_at) FROM stdin;
118+
\.
119+
120+
121+
--
122+
-- Data for Name: users; Type: TABLE DATA; Schema: public; Owner: admin
123+
--
124+
125+
COPY public.users (id, username, email, created_at, phone) FROM stdin;
126+
1 mark mark@example.com 2026-07-18 11:45:54.215925 09123456789
127+
2 juan juan@example.com 2026-07-18 11:45:54.215925 09987654321
128+
\.
129+
130+
131+
--
132+
-- Name: tasks_id_seq; Type: SEQUENCE SET; Schema: public; Owner: admin
133+
--
134+
135+
SELECT pg_catalog.setval('public.tasks_id_seq', 1, false);
136+
137+
138+
--
139+
-- Name: users_id_seq; Type: SEQUENCE SET; Schema: public; Owner: admin
140+
--
141+
142+
SELECT pg_catalog.setval('public.users_id_seq', 2, true);
143+
144+
145+
--
146+
-- Name: tasks tasks_pkey; Type: CONSTRAINT; Schema: public; Owner: admin
147+
--
148+
149+
ALTER TABLE ONLY public.tasks
150+
ADD CONSTRAINT tasks_pkey PRIMARY KEY (id);
151+
152+
153+
--
154+
-- Name: users users_pkey; Type: CONSTRAINT; Schema: public; Owner: admin
155+
--
156+
157+
ALTER TABLE ONLY public.users
158+
ADD CONSTRAINT users_pkey PRIMARY KEY (id);
159+
160+
161+
--
162+
-- PostgreSQL database dump complete
163+
--
164+
165+
\unrestrict YWhz4ALTeDQK2TekRCVweACNkZvAWoY2Ni1yU5HfULcpAYfR8ccbGgFwSoWW7we
166+

‎labs/databases/postgresql-lab/migrations/001_create_users.sql‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
CREATE TABLE users (
2+
id SERIAL PRIMARY KEY,
3+
username TEXT NOT NULL,
4+
email TEXT NOT NULL,
5+
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
6+
);

‎labs/databases/postgresql-lab/migrations/002_create_tasks.sql‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
CREATE TABLE tasks (
2+
id SERIAL PRIMARY KEY,
3+
title TEXT NOT NULL,
4+
completed BOOLEAN DEFAULT FALSE,
5+
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
6+
);

‎labs/databases/postgresql-lab/migrations/003_add_phone_to_users.sql‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
ALTER TABLE users
2+
ADD COLUMN phone TEXT;

0 commit comments

Comments
 (0)