Orion v2 (#49) (#101)

* Initial commit

* Initial orion draft

* Hide private data

* Fix extendedChannels query

* docker-compose db config adjustments

* Fix ExtendedBasicChannelFields

* Add `video` view

* Storage, Membership and ChannelCreated mappings

* Channel events processing

* Video mappings + some generic fixes

* Mappings: Comments, reactions + some generic fixes

* Cleanup

* NFT mappings, some post-testing fixes

* Video views mutation

* Channel follows + video views improvements

* Channel/video reports

* Category filtering: Part 1

* State subscription

* Category filtering: Part 2 + some refactorization

* Category filtering: Part 3 - add missing views and operator auth

* mostViewedVideosConnection custom query + generic custom queries fixes

* Kill switch

* mostRecentChannels query

* channelNftCollectors query

* extendedVideoCategories query

* Fix AND-OR queries and "eq: null"

* Docker setup improvements

* Video hero

* Category featured videos

* Entity caching (overlay) refactorization

* Prettier

* Documentation, setup improvements, cleanup

* FIX: Save NextEntityIds

* Excludable content

* - Allow limiting the total number of cached entities
- Deployment manifest fixes
- Custom ARCHIVE_GATEWAY_URL instead of HOST/PORT
- Updated dependencies
- `processAssets` bug fix

* setVideoViewPerIpTimeLimit operator mutation

* Ephesus mappings WIP

* Fix conflicts with local Joystream docker setup

* Metadata-protobuf patch location fix

* Ephesus Atlas queries

* Update CHANGELOG

* Linting/formatting fixes, dead code removal

* Apps metaprotocol mappings

* Open auction topBid fix

* Fix metadata-protobuf patch

* Add signAppActionCommitment endpoint

* Tests: Basic data comparison script + bug fixes

* Remove channel-owned apps

* Remove Ephesus scope

* Apps as member-owned initial rework

* Fixes after compareState vs mainnet

* Remove lead-owned apps, fix typeorm-codegen patch

* AppAction support (Joystream/joystream#4631)

* Update CHANGELOG based on tests

* Allow controlling "is_censored" content exclusion via env

* buildExtendedChannelsQuery fix (activeVideosCount_gt: 0)

* Benchmarking script

* Benchmarking results

* Event queries optimalizations, fix gitignored v1 and v2 schemas

* Adjust autovacuum_analyze_scale_factor

* Fix entity cache cleanup

* Add index on event's `inExtrinsic` field

* Benchmarking round 2

* Fix req.ip reverse proxy issue, allow conditionally displaying excluded content in results

* Operator queries

* Update docs

* Prettier format

* Add CI checks

* Remove issue template, fix "workflows" location

* Disallow app deletion

* Update `generateAppActionCommitment` from `@joystream/js`

* Fix channel follows, video views and reports after operator queries

* Fix #82

* Comment notifications: Don't notify the author about their own comment

* Introduce a mechanism to preserve views, follows, reports and config data when updating the processor

* mostViewedVideosConnection missing params fix

* Fix assets encoding (AppAction)

* Assets url resolving

* Remove accidently commited file

* Fix Aquarium deployment

* Update CHANGELOG

* Improve caching speed and add more logs

* Latest apps-related updated (generateAppActionCommitment, mappings)

* Add Request entity

* Add mutation to request featured nft

* Add new flag for nft

* Add protected mutation to edit featured NFTs

* Small fixes

* Regenerating migration

* Hiding request table

* Process member banning/unbanning

* Fix: Remove banned members when channel is removed

* Update migration

* State queries: Re-enable apps

* Views: Add missing chanel/video related entities

* Excluding/restoring comments: Update:
- parentComment.repliesCount,
- parentComment.reactionsAndRepliesCount,
- video.commentsCount

* Operator queries: 401 Unauthorized, remove `x-display-hidden-entities` (operator just sees all hidden entities by default)

* Update changelog

* Simplify processor state subscription

* Prevent the same comment from being "excluded"/"restored" multiple times

* Small adjustments to docs and compareState script

* Orion v1 migration scripts

* New logic for comment exclusion

* Fix: Reference to `yarn` in package.json

* Increase interval of ProcessorStateRetriever from 100 ms to 1000 ms

* Dependencies update: Fix npm audit issues

* Remove cached gitignore files

* CR fixes

* Delete index.ts

* @subsquid/graphql-server `forbidUnknownValues` patch

* Asset resolution: Return multiple urls

* Update changelog

* FIX: Events cleanup on deleteVideo

---------

Co-authored-by: WRadoslaw <r.wyszynski00@gmail.com>

Author: Lezek123

.dockerignore

@@ -1,5 +1,4 @@
-*.log
-node_modules/
-dist/
-Dockerfile
-docker-compose.yml
+/.git
+/node_modules
+/lib
+/*Versions.jsonl

.env

@@ -0,0 +1,32 @@
+# LOCAL ENVIRONMENT
+
+# Db config
+DB_NAME=squid
+DB_PASS=squid
+DB_PORT=23798
+# Processor service host
+PROCESSOR_HOST=localhost
+# Processor service prometheus port
+PROCESSOR_PROMETHEUS_PORT=3337
+# Graphql server port
+GQL_PORT=4350
+
+# Archive gateway url
+ARCHIVE_GATEWAY_URL=${CUSTOM_ARCHIVE_GATEWAY_URL:-http://localhost:8888/graphql}
+
+# Default config values
+SUPPORT_NO_CATEGORY_VIDEOS=true
+SUPPORT_NEW_CATEGORIES=true
+KILL_SWITCH_ON=false
+VIDEO_VIEW_PER_IP_TIME_LIMIT=86400 # 86400 seconds = 24 hours
+# Operator API secret
+OPERATOR_SECRET=this-is-not-so-secret-change-it
+# Processor configuration
+MAX_CACHED_ENTITIES=1000
+APP_PRIVATE_KEY=this-is-not-so-secret-change-it
+
+# Adjust accordingly with the number of trusted (used) reverse proxies!
+TRUSTED_REVERSE_PROXIES=1
+
+# Debug settings
+SQD_DEBUG=api:*