Refactoring Handlers and Business Logic, Middleware into Service Architecture

.gitignore

@@ -55,3 +55,7 @@ tmp/
 
 # Apple Silicon Docker Compose Override
 docker-compose.override.yml
+
+
+# vs code
+.vscode/

README/CHECKPOINTS/CHECKPOINT20251020-51.md

@@ -0,0 +1,369 @@
+SocialPredict Move-Only Refactor (Handlers → Thin; Domain/Repo Split)
+
+## Goal
+Refactor the backend so **handlers/** contain only HTTP glue (JSON in/out, status codes, error mapping). Move all non-HTTP logic out into **internal/domain/** and **internal/repository/** without changing function bodies or behavior. Add wiring in **internal/app**. Prepare for future microservices and OpenAPI-first workflow.
+
+## Constraints
+- **No behavior changes**. *Move code only*. Adjust imports and package names as needed.
+- Do **not** modify models/DB schema/migrations.
+- Handlers must not import `gorm.io/gorm` or manipulate DB directly after refactor.
+- Preserve existing tests; add lightweight guard checks.
+
+---
+
+## Target Layout (backend/)
+backend/
+handlers/ # HTTP-only (keep)
+admin/
+bets/
+cms/
+markets/
+dto/ # HTTP request/response types (JSON)
+math/
+metrics/
+positions/
+setup/
+stats/
+tradingdata/
+users/
+
+internal/
+domain/ # pure business logic (no HTTP, no GORM)
+admin/
+bets/
+cms/
+markets/
+metrics/
+positions/
+stats/
+tradingdata/
+users/
+repository/ # data access (GORM or future HTTP client adapters)
+admin/
+bets/
+cms/
+markets/
+metrics/
+positions/
+stats/
+tradingdata/
+users/
+app/
+container.go # composition root: repos -> services -> handlers
+validation/ # shared validators (optional)
+math/ # (optional) if you move wpam/dbpm out of handlers
+probabilities/
+wpam/
+dbpm/
+
+models/
+migration/
+setup/
+logger/
+security/
+util/
+
+README/BACKEND/API/
+openapi.yaml
+API-DOCS.md
+API-DESIGN-REPORT.md
+
+markdown
+Copy code
+
+---
+
+## High-Level Steps
+
+1) **Create new directories** under `backend/internal/{domain,repository,app}` and `handlers/*/dto`.
+2) **Move non-HTTP code** out of each `handlers/<area>`:
+   - DB access → `internal/repository/<area>`
+   - Business logic/pure functions → `internal/domain/<area>`
+   - Leave handlers with: parse request, call service, map errors, write response.
+3) **Adjust imports** to new package paths. Do not change function bodies except for package/name/imports.
+4) **Add wiring** in `internal/app/container.go` to construct repos → services → handlers. Handlers depend on service interfaces, not repos directly.
+5) **DTO separation**: Request/response structs used only for HTTP live in `handlers/<area>/dto`.
+6) **Scaffold OpenAPI docs** in `README/BACKEND/API/` (placeholders OK now).
+7) **Run checks** and fix any remaining direct DB usage in handlers.
+8) **Build & test**.
+
+---
+
+## Concrete Tasks (execute in order)
+
+### 0) Sanity
+- `go mod tidy`
+- `go test ./...`
+
+### 1) Create directories
+- Create all directories from the Target Layout that don’t exist.
+
+### 2) For each package in `handlers/`:
+- Identify files that import **gorm** or **models** or contain business rules.
+- **Move them** to:
+  - `internal/repository/<area>` if they touch GORM/DB.
+  - `internal/domain/<area>` if they implement business logic/pure functions.
+- If a handler file mixes HTTP and DB/logic, **split** it:
+  - Keep HTTP parts in `handlers/<area>`.
+  - Move the rest accordingly.
+
+### 3) DTOs
+- Move HTTP request/response structs to `handlers/<area>/dto` (keep JSON tags here).
+- Domain structs should have **no JSON/GORM tags**.
+
+### 4) Wiring (create) — `backend/internal/app/container.go`
+Paste the following skeleton and adapt names to your code:
+
+```go
+package app
+
+import (
+	"time"
+	"gorm.io/gorm"
+
+	"socialpredict/setup"
+
+	// Domain and repository packages (adjust imports to your actual names)
+	dmarkets "socialpredict/internal/domain/markets"
+	rmarkets "socialpredict/internal/repository/markets"
+	hmarkets "socialpredict/handlers/markets"
+)
+
+type Clock interface{ Now() time.Time }
+type sysClock struct{}
+func (sysClock) Now() time.Time { return time.Now() }
+
+// BuildMarkets wires markets repository -> service -> handler.
+// Add more builders for users, bets, positions, stats, etc.
+func BuildMarkets(db *gorm.DB, econ *setup.EconomicConfig) *hmarkets.Handler {
+	repo := rmarkets.NewGormRepository(db)
+	svc  := dmarkets.NewService(repo, sysClock{}, dmarkets.Config{
+		// example validation hook; keep optional
+		ValidateLabel: func(s string) bool { return len(s) >= 1 && len(s) <= 20 },
+		// inject economics/config here if needed by service
+		Econ: econ,
+	})
+	return hmarkets.NewHandler(svc)
+}
+In your server startup, call app.BuildMarkets(db, econ) and register routes. Repeat for users/bets/positions/stats.
+
+5) Handlers use service interfaces only
+In each handlers/<area>, expose a NewHandler(service Interface) constructor.
+
+Replace direct DB calls with service calls.
+
+Example minimal HTTP-only handler (adjust router specifics):
+
+go
+Copy code
+package markets
+
+import (
+	"encoding/json"
+	"net/http"
+	"strconv"
+
+	dmarkets "socialpredict/internal/domain/markets"
+	"socialpredict/handlers/markets/dto"
+)
+
+type Service interface {
+	SetCustomLabels(ctx context.Context, marketID int64, yes, no string) error
+	// add other domain methods here
+}
+
+type Handler struct{ svc Service }
+
+func NewHandler(svc Service) *Handler { return &Handler{svc: svc} }
+
+func (h *Handler) PutLabels(w http.ResponseWriter, r *http.Request) {
+	idStr := r.PathValue("id") // adapt for your router
+	id, err := strconv.ParseInt(idStr, 10, 64)
+	if err != nil { http.Error(w, "invalid id", http.StatusBadRequest); return }
+
+	var body dto.LabelRequest
+	if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
+		http.Error(w, "invalid json", http.StatusBadRequest); return
+	}
+
+	if err := h.svc.SetCustomLabels(r.Context(), id, body.YesLabel, body.NoLabel); err != nil {
+		switch err {
+		case dmarkets.ErrMarketNotFound:
+			http.Error(w, "not found", http.StatusNotFound)
+		case dmarkets.ErrInvalidLabel:
+			http.Error(w, "invalid label", http.StatusBadRequest)
+		default:
+			http.Error(w, "internal error", http.StatusInternalServerError)
+		}
+		return
+	}
+
+	w.WriteHeader(http.StatusNoContent)
+}
+6) Domain & Repository skeletons
+Domain (internal/domain/markets/service.go):
+
+go
+Copy code
+package markets
+
+import (
+	"context"
+	"errors"
+	"time"
+	"socialpredict/setup"
+)
+
+var (
+	ErrMarketNotFound = errors.New("market not found")
+	ErrInvalidLabel   = errors.New("invalid label")
+)
+
+type Config struct {
+	ValidateLabel func(string) bool
+	Econ *setup.EconomicConfig
+}
+
+type Clock interface{ Now() time.Time }
+
+type Repository interface {
+	GetByID(ctx context.Context, id int64) (*Market, error)
+	UpdateLabels(ctx context.Context, id int64, yes, no string) error
+}
+
+type Service struct {
+	repo  Repository
+	clock Clock
+	cfg   Config
+}
+
+func NewService(repo Repository, clock Clock, cfg Config) *Service {
+	return &Service{repo: repo, clock: clock, cfg: cfg}
+}
+
+type Market struct {
+	ID          int64
+	Question    string
+	OutcomeType string
+	YesLabel    string
+	NoLabel     string
+}
+
+func (s *Service) SetCustomLabels(ctx context.Context, id int64, yes, no string) error {
+	if s.cfg.ValidateLabel != nil {
+		if !s.cfg.ValidateLabel(yes) || !s.cfg.ValidateLabel(no) {
+			return ErrInvalidLabel
+		}
+	}
+	if _, err := s.repo.GetByID(ctx, id); err != nil {
+		return ErrMarketNotFound
+	}
+	return s.repo.UpdateLabels(ctx, id, yes, no)
+}
+Repository (internal/repository/markets/repo.go):
+
+go
+Copy code
+package markets
+
+import (
+	"context"
+	"errors"
+
+	"gorm.io/gorm"
+	"socialpredict/models"
+	dmarkets "socialpredict/internal/domain/markets"
+)
+
+type GormRepository struct{ db *gorm.DB }
+func NewGormRepository(db *gorm.DB) *GormRepository { return &GormRepository{db: db} }
+
+func (r *GormRepository) GetByID(ctx context.Context, id int64) (*dmarkets.Market, error) {
+	var m models.Market
+	if err := r.db.WithContext(ctx).First(&m, id).Error; err != nil {
+		if errors.Is(err, gorm.ErrRecordNotFound) {
+			return nil, dmarkets.ErrMarketNotFound
+		}
+		return nil, err
+	}
+	return &dmarkets.Market{
+		ID:          m.ID,
+		Question:    m.QuestionTitle,
+		OutcomeType: m.OutcomeType,
+		YesLabel:    m.YesLabel,
+		NoLabel:     m.NoLabel,
+	}, nil
+}
+
+func (r *GormRepository) UpdateLabels(ctx context.Context, id int64, yes, no string) error {
+	return r.db.WithContext(ctx).Model(&models.Market{}).
+		Where("id = ?", id).
+		Updates(map[string]any{
+			"yes_label": yes,
+			"no_label":  no,
+		}).Error
+}
+7) OpenAPI docs scaffolding
+Create files:
+
+README/BACKEND/API/openapi.yaml (put a minimal valid OAS 3.0.3 skeleton)
+
+README/BACKEND/API/API-DOCS.md (index + how to run Swagger UI)
+
+README/BACKEND/API/API-DESIGN-REPORT.md (bullets of current state + roadmap)
+
+Keep these as placeholders now; full authoring can be a separate task.
+
+8) Guard checks (post-move)
+Run these to ensure no DB/ORM leaks remain in handlers:
+
+bash
+Copy code
+# 1) No GORM in handlers
+! grep -R --line-number --ignore-case 'gorm\.io/gorm' backend/handlers || (echo "GORM import found in handlers!"; exit 1)
+
+# 2) No models import in handlers
+! grep -R --line-number --fixed-strings 'socialpredict/models' backend/handlers || (echo "models used in handlers!"; exit 1)
+
+# 3) No SQL strings in handlers (heuristic)
+! grep -R --line-number -E '\b(SELECT|UPDATE|INSERT|DELETE)\b' backend/handlers || (echo "SQL detected in handlers!"; exit 1)
+9) Build & test
+bash
+Copy code
+go mod tidy
+go build ./...
+go test ./...
+Exit Criteria (must all pass)
+Handlers compile with no gorm or models imports.
+
+All unit tests pass unchanged.
+
+Server builds and routes resolve via internal/app/container.go.
+
+DTOs live under handlers/<area>/dto.
+
+OpenAPI files exist in README/BACKEND/API/ (validated later).
+
+Microservices Readiness (what this enables)
+Each <area> has:
+
+internal/domain/<area> defining interfaces (ports).
+
+internal/repository/<area> implementing adapters (GORM today).
+
+Later, to split a service (e.g., positions):
+
+Create a new service with its own Dockerfile and DB.
+
+Generate an HTTP/gRPC client from openapi.yaml.
+
+Replace the GORM repo with a client repo implementing the same domain interface.
+
+Flip binding in internal/app/container.go based on env (USE_REMOTE_POSITIONS=true).
+
+Notes
+Keep commit granularity: one directory at a time (easier to review).
+
+If a handler still needs a tiny helper, prefer placing it in domain and calling it—avoid logic drift back into HTTP.
+
+Prefer sentinel errors in domain; map to HTTP codes in handlers.