Page Pulse

Give it a URL, it fetches the page server-side and tells you what's wrong with it: SEO fields, content metrics, security headers, cookie flags, mixed content, TLS certificate, and a letter grade on top.

Live: https://page-pulse-v4ca.onrender.com/ Code: https://github.com/1PoPTRoN/page-pulse

Python 3.11+, FastAPI, httpx, selectolax. No database, no build step, no frontend framework.

Setup

pip install -r requirements-dev.txt
uvicorn app.main:app --reload      # http://127.0.0.1:8000
pytest                             # 139 tests, ~0.3s, no network access

requirements.txt is the runtime set (4 packages, all pinned). requirements-dev.txt adds pytest and respx on top.

The test suite never touches the network. If it fails, something actually broke.

For the things tests can't cover (real DNS, real TLS handshakes, the browser UI, the deployed instance) there's a manual battery in docs/MANUAL_TESTING.md.

How it's laid out

app/fetch/       validates the URL, fetches it, builds a PageSnapshot
app/analyzers/   read the snapshot, each returns one report section
app/scoring/     turns sections into a grade
app/api/         wires it together
app/core/        config, error types, rate limiter

Dependencies only point one direction. app/fetch/ knows about sockets and nothing about HTML. The analyzers know about HTML and nothing about sockets. PageSnapshot is the wall between them.

Adding a check: write a function that takes a PageSnapshot and returns a dict, add one line to ANALYZERS in app/analyzers/registry.py. Add a Penalty row in app/scoring/grade.py if it should affect the grade. That's it.

Each analyzer runs in its own try/except. If one blows up you get {"error": "analysis_failed"} for that section and the rest of the report still renders. A partial report beats a 500.

API

POST /api/v1/audit

{"url": "https://example.com"}

url is required, 1 to 2048 chars. Request body capped at 16 KB.

200 response:

{
  "target": {
    "requested_url": "https://example.com",
    "final_url": "https://example.com/",
    "status_code": 200,
    "elapsed_ms": 412,
    "content_type": "text/html; charset=utf-8",
    "content_length": 1256,
    "redirect_chain": [{"url": "https://example.com", "status": 301}]
  },
  "seo": {"title": "Example Domain", "meta_description": null, "h1_count": 1},
  "content": {"word_count": 137, "images_total": 4, "images_missing_alt": 2,
              "likely_client_rendered": false},
  "security_headers": {
    "hsts": {"present": true, "value": "max-age=31536000"},
    "csp": {"present": false, "value": null},
    "x_frame_options": {"present": false, "value": null},
    "x_content_type_options": {"present": false, "value": null},
    "referrer_policy": {"present": false, "value": null},
    "permissions_policy": {"present": false, "value": null}
  },
  "cookies": {"total": 2, "insecure": [{"name": "session", "missing": ["Secure", "SameSite"]}]},
  "mixed_content": {"count": 1, "resources": ["http://cdn.example.com/a.js"]},
  "tls": {"valid": true, "issuer": "Let's Encrypt", "subject": "example.com",
          "expires_at": "2026-09-14", "days_remaining": 51, "expiring_soon": false},
  "grade": {
    "score": 72, "letter": "C",
    "penalties": ["missing_csp", "insecure_cookie"],
    "penalty_points": {"missing_csp": 15, "insecure_cookie": 15},
    "max_points": 175,
    "skipped_checks": []
  }
}

Conventions I stuck to:

• Missing value is null, never a missing key. You should never have to guess whether a field is absent because the page lacks it or because my code broke.
• Analyzer blew up: {"error": "analysis_failed"}.
• Analyzer couldn't run: {"skipped": "<reason>"}.
• A header sent with an empty value reports "present": false. An empty CSP enforces nothing, so calling it present would be a lie.
• HSTS over plain http reports "present": false plus "ignored_over_http": true, because browsers throw that header away when it doesn't arrive over TLS.
• Repeated header adds "duplicated": true and a "values" list. Browsers apply the intersection of repeated CSP headers, so showing one value would misrepresent it.

What counts as HTML: text/html, application/xhtml+xml, XML-served markup. If there's no Content-Type at all I sniff for a leading <!doctype html> instead of giving up.

Non-HTML (PDF, image, JSON) returns 200, not an error. target, security_headers, cookies and tls all still work, because their inputs are headers, not markup. The HTML-dependent sections come back {"skipped": "non_html_content"}. Erroring out would throw away real findings I already have.

Errors

{"error": {"code": "upstream_timeout", "message": "Target did not respond within the deadline"}}

Code	HTTP	When
invalid_url	400	Unparseable, no hostname, over 2048 chars, or scheme isn't http/https
blocked_target	403	Resolved to an address the guard refuses
upstream_unreachable	502	DNS failure or connection refused
certificate_invalid	502	Reached the host, its certificate failed verification
too_many_redirects	502	More than 5 hops
upstream_timeout	504	Blew the fetch deadline
response_too_large	413	Body went over 5 MB mid-stream
request_too_large	413	Request body over 16 KB
rate_limited	429	Over the allowance
internal_error	500	Something I didn't anticipate. Traceback goes to the log, not to you

FastAPI returns 422 in its own shape when url is missing or too long.

Error messages never contain a resolved IP address. More on why below.

GET /health

{"status": "ok"}.

Three design decisions

1. The SSRF guard runs at the socket, not on the URL string

This service fetches whatever URL you hand it. That's the whole product, and it's also the entire security problem. Everything else follows from it.

The obvious version is: parse the URL, check the hostname isn't localhost or 10.x, then fetch. That version is broken twice over.

First, redirects. Validate https://evil.com, hand it to httpx with redirects on, and evil.com 302s you to 169.254.169.254. The guard never sees it. So I turned redirect following off and walk the chain myself, re-running the full guard on every hop.

Second, and this one took me longer to accept: even with redirects handled, I was resolving DNS twice. Once in the guard to check the address, once inside httpx when it opened the socket. A DNS server you control can answer differently the second time. My README used to list this as a known limitation and say fixing it would break TLS SNI. That was wrong. The guard now hands back the addresses it approved and I pin the connection to them through a custom httpcore backend. Because the pinning happens at the socket layer instead of by rewriting the URL, the Host header and the SNI name are untouched and certificate verification still works normally.

I also stopped putting the resolved IP in the error message. blocked_target used to say "resolves to non-public address 10.1.2.3", which is a free internal DNS lookup for anyone who asks. It goes to the log now.

Cost of this decision: the fetch path is more code than httpx.get(url) and it leans on one httpcore internal (_pool._network_backend). That's why the dependencies are pinned.

2. PageSnapshot is a hard wall, and analyzers are pure functions of it

Every network concern is finished before a single analyzer runs. Analyzers get a frozen picture and return a dict. No I/O, no globals, no mutating the snapshot.

I didn't do this for tidiness. I did it because I wanted the test suite to be fast enough that I'd actually run it, and network-dependent tests are neither fast nor trustworthy. 139 tests run in about 0.3 seconds with the network unplugged.

The bit I'd argue for in an interview: "analyzers are pure" is worthless as a docstring claim, so there's a test that runs the whole registry twice over one snapshot and asserts both identical output and an unmodified snapshot. If someone adds an analyzer that reaches out to the network, that test fails.

Cost: the same HTML gets parsed by three different analyzers instead of once. I know. On a 5 MB page that's real waste and I'd fix it by caching the parse tree on the snapshot if this were going to grow.

3. The grade is a percentage of what I could actually check

My first version scored out of a flat 100 and skipped checks it couldn't run. Looked reasonable. Then I ran it against a PDF and got an A, and against react.dev and got a D.

The PDF scored better because skipping a check removed the penalty but left the total at 100. Give the grader less to find and the score goes up. Backwards.

Worse, an unencrypted http site could score a perfect 100. There was no penalty for not having HTTPS at all, and the site could still collect points for security headers. A tool that hands an A to a plaintext site is worse than useless, because someone might believe it.

So: skipping a check now removes its weight as well as its penalty. max_points tells you how much was actually assessable and skipped_checks names what wasn't. A C computed from 175 points means something different from a C computed from 95, and now you can see which you got. not_https is the heaviest single penalty at 25 because serving over plaintext defeats every other control at once.

Same plaintext site now scores F 37. The PDF scores B 83 out of 120 possible, with 5 checks listed as skipped.

Cost: score is no longer comparable across pages that had different checks run, which is a real downside. I decided a number you can interpret beats a number that's uniform and wrong.

Scoring table

Score starts at 100% of assessable points. 90+ A, 80+ B, 70+ C, 60+ D, else F.

Penalty	Points
not_https	25
missing_title	20
certificate_invalid	20
missing_hsts	15
missing_csp	15
insecure_cookie	15
mixed_content	15
missing_x_frame_options	10
images_missing_alt	10
missing_x_content_type_options	5
missing_referrer_policy	5
missing_permissions_policy	5
missing_meta_description	5
bad_h1_count	5
certificate_expiring_soon	5

A missing title is the worst SEO defect you can have, so it's weighted above any single header. An insecure cookie sits level with a missing CSP because it's an actual credential exposure, not a missing mitigation.

Same input always produces the same grade. grade() is a pure function with no clock and no randomness in it.

Limits

All in app/core/config.py, none hardcoded anywhere else:

• connect 5s, read 10s, total 15s
• certificate probe bounded separately at 5s
• body cap 5 MB, enforced while streaming (the download aborts, it doesn't finish and then get measured)
• 5 redirect hops
• 10 requests / 60s per client, over a table capped at 10,000 clients
• 8 outbound fetches in flight process-wide

What it doesn't do

• No JavaScript. I fetch static HTML, so a React SPA gives me the pre-hydration shell. I detect the likely case (scripts present, almost no prose) and flag it as likely_client_rendered rather than reporting "3 words" like it's a real answer. Fixing it properly needs a headless browser, which is a different service with a different bill.
• The rate limit depends on a deployment assumption. I read client identity from the right-hand end of X-Forwarded-For, counting in TRUSTED_PROXY_HOPS entries, because the left end is whatever the caller typed. Set that number wrong and identity is either forgeable or collapses to the proxy. The thing that actually bounds outbound traffic is the fetch semaphore, which no header can touch.
• Rate limit state lives in process memory, so the service runs on one worker. Multiple instances would need Redis.
• No certificate details for a certificate that fails verification. I report the reason (certificate_expired, hostname_mismatch, self_signed_certificate, incomplete_chain) but not the issuer or expiry, because reading those means completing a handshake I'm refusing on purpose.
• Cold start. Free tier spins down. First request after idle takes 30 to 60 seconds.

AI usage

I used Claude throughout: to scaffold modules, to learn the parts of SSRF defence I hadn't implemented before, and then to audit the finished code and pressure-test my own reasoning. The audit is where it earned its keep. It found that my guard missed RFC 6598 CGNAT space (100.64.0.0/10), that I was resolving DNS twice and leaving a rebinding window, that my error messages leaked internal IPs, that an empty security header scored as a pass, that cookies set on redirect hops were being dropped, and that an http-only site could score an A. I fixed all of those and wrote a regression test for each. What I did not do is trust it: I ran every claim before acting on it, which caught it being wrong about my Python version being vulnerable, and two of the fixes it helped write were broken on the first attempt (a stray "error": None key that silently switched off certificate scoring, and an exception chain walk one level too shallow to spot an expired certificate). Both surfaced only when I tested against real sites instead of reading the diff, which is why there's a badssl.com battery in the manual testing guide now.

---

Built for Digital Heroes Training Task