Skip to content
/ itvault Public

Web application designed to store source code intelligence and provide option to extract it on demand.

License

Notifications You must be signed in to change notification settings

trolit/itvault

Repository files navigation

Memory is the mother of all wisdom.
-Aeschylus


itvault is designed to:

  1. Keep what each part of code is used for.
  2. Extract selected parts on demand (with maintained files structure).
Quick introduction

App relies on simple mechanism - coloring files content. After uploading project files, permitted user(s) create blueprints - abstract elements to group code. Then user(s) with appropriate permission can use them to mark code:

file colored with Group1 blueprint file colored with Group2 blueprint
formatDate.ts colored with Group1 blueprint formatDate.ts colored with Group2 blueprint

Users with access to these workspaces (= projects) can also obtain permission to generate bundles - extracts of project consisting only of selected parts (with maintained hierarchy). Result of generating bundle with blueprints: Group1 and Group2:

file content in generated bundle

For short demonstration of coloring & features extraction, refer to youtube video

Please note that although app is in advanced stage (spent 1+ year), it's not complete solution. If you would like to use it (or it's parts), I'd recommend to only take backend as Vue's Composition API was taken to experiment with new approach and it still needs big refactoring. PS: I see potential in Comp API - after working out with form submit - but still prefer Options API as it requires less refactoring to achieve something fancy (at least from my POV).

Stack

Backend
  • TypeScript: 4.9
  • Node: v18.17
  • Web framework: Express.js
  • ORM framework: TypeORM
  • Logger: winston
  • DI: tsyringe
  • Tests: Mocha (nyc, chai, sinon, supertest, testcontainers)
  • File storage: local or AWS S3 (simulated with LocalStack)
  • Secondary DB (store for sessions/roles): Redis
  • Queues: RabbitMQ
  • Schema validation: yup
  • Client-Server real-time communication: Engine.io
  • Mailing: mustache (renderer), nodemailer (sender), maildev (local testing)
  • Authentication: currently hybrid, half on server (Redis), half on client (JWT)
Frontend
  • TypeScript: 4.7
  • Framework: Vue 3 (experimenting with Composition API)
  • State management: Pinia
  • Components library: Naive UI
  • Forms library: vee-validate (+yup)
  • Charts library: apexcharts
  • CSS: PostCSS
  • E2E tests: Cypress

Running

Development
  1. Install dependencies (npm install).

  2. Create .env files from .env.example.

  3. Initialize docker containers from backend dir.

    npm run dc:dev:start
  4. Prepare database (migrations, seeders).

    npm run db:setup
  5. Start backend.

    npm run dev

    *To handle bundle generation & mail sending, run queues:

    npm run queues:dev
  6. Start frontend.

    npm run dev
    # Super user (all permissions)
    email: [email protected]
    password: 1234
    
    # Member
    email: [email protected]
    password: 1234

Note that default setup uses local files storage. To check out AWS S3 (through LocalStack):

  1. Stop backend.

  2. Set FILES_STORAGE_MODE env variable to aws.

  3. Start LocalStack.

    npm run dc:localstack:start
  4. Get awslocal tool and setup bucket (check README).

  5. Start backend (npm run dev).

  6. Upload some files to see effect:

awslocal s3api list-objects --bucket itvault-bucket
{
    "Contents": [
        {
            "Key": "workspace-6/b6a66d7d95ad9b2b585dc0700.txt",
            "LastModified": "2024-03-22T10:40:19.000Z",
            "ETag": "\"02003d02d9dab50aead91ee0ddad97ed\"",
            "Size": 710,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "webfile",
                "ID": "75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a"
            }
        }
    ],
    "RequestCharged": null
}

Disclaimers:

  • Files created through seeders won't be reachable.
  • When running free version of LocalStack, data is not persistent.
Production
  1. Initialize docker containers from backend dir.

    npm run dc:prod:start
  2. Set NODE_ENV to production in backend.

  3. Prepare database (migrations, seeders).

    npm run db:setup
  4. Create super user account manually or create production seeder.

  5. Start backend.

    npm run prod

    *To handle bundle generation & mail sending, run queues:

    npm run queues:prod

Testing

Unit tests
npm run test:unit
Integration tests
# NODE_ENV=test
npm run testcontainers:up
npm run test:integration
npm run testcontainers:down
E2E tests
# 1. Setup
# (backend)
# NODE_ENV=test
npm run testcontainers:up
npm run db:setup

# (frontend)
npm run build # or build-only

# 2. Run
npm run test:e2e

# 3. Clean up
npm run testcontainers:down

Features

  • store files locally or in cloud
  • [role] - [permission] granular access system
  • global chat with configurable depth
  • mail system with mustache and nodemailer
  • hybrid API versioning (branch ✅, query ✅, leaf ⛔)
  • real-time client-server communication with Engine.io
  • workspace insights
  • strong typing (e.g. schema <-> controller relation)

Plus sessions management, seeders (via typeorm-extension), hybrid authentication - half on client (JWT), half on server (Redis)

To consider/To do

  • rework all "ids" to UUID not only to improve security but simplify integration tests
  • create modular variant of project
  • ability to preview users of specific role
  • mobile variant of client
  • create separate app for management (admin)
  • finish real-time communication events
  • listen to release pull requests (develop -> main) and then create new variants according to PR data
  • check if there is option to train AI of workspace code to determine which parts refer to which blueprint
  • ability to edit file content while maintaining colorings structure at the same time
  • ability to preview few lines of version when reviewing files for Bundle generation
  • job that removes files if they do not have entry in database
  • monitor status of each endpoint
  • define "parent" permission (can't update children permissions if parent is disabled)
  • Workspace - Files tab - rename/delete directory
  • Workspace - Files tab - icons matching file extension

Relevant posts

Links

About

Web application designed to store source code intelligence and provide option to extract it on demand.

Topics

Resources

License

Stars

Watchers

Forks