Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load index into memory in download-files #3762

Merged
merged 1 commit into from
Mar 27, 2024
Merged

Load index into memory in download-files #3762

merged 1 commit into from
Mar 27, 2024

Conversation

Michal-Leszczynski
Copy link
Collaborator

Fixes #3761

@Michal-Leszczynski
Copy link
Collaborator Author

Command output after fixes:

root@be9c501bd208:/# scylla-manager-agent download-files -L s3:restoretest-upload-ks  -n 354b6aa5-1b1b-416e-a5eb-8a41def31d46  -T sm_20231212164821UTC  --dump-manifest
{
  "version": "v2",
  "cluster_name": "myc1",
  "ip": "192.168.200.32",
  "size": 210173,
  "tokens": [
    -9208126688470436482,
    -9185110959739019928,
    -9137502779526004301,
    -9072725176422645021,
    -8884492911237353876,
    -8878434788836566161,
    -8821529089164233297,
    -8655399579400223039,
    -8620264064638984058,
    -8560355716948062204,
    -8523268607462530506,
    -8506013102016545154,
    -8391680983204486927,
    -8342515940325461319,
    -8338763071613601785,
    -8265905289747798690,
    -8201448194644581913,
    -8074944618794301626,
    -7834480372990351228,
    -7785640251224882232,
    -7765346515825318939,
    -7631832182078232007,
    -7502940169108976584,
    -7433222376505733641,
    -6983764526925757614,
    -6876850327801427538,
    -6768083445676152666,
    -6740508957885329303,
    -6674363182243296499,
    -6628643327096210497,
    -6588160392324151851,
    -6536496666095378640,
    -6455877879869492920,
    -6436495346092191569,
    -6428064465473236071,
    -6237532661097738984,
    -6184490222398052119,
    -6173321313061377615,
    -6077489929980561273,
    -5991011635080767190,
    -5906732077847432488,
    -5891676592125660152,
    -5860524508167449756,
    -5817096821989031806,
    -5630812369779735645,
    -5619022598734544346,
    -5398641193048153570,
    -5351299532673858414,
    -5240463355619462801,
    -5174208291531876374,
    -4994467698408364251,
    -4964011481906975775,
    -4935752025224112661,
    -4924284901089498247,
    -4924146191887721718,
    -4914410827151027317,
    -4885174128911954799,
    -4810947957609932427,
    -4810335977963289240,
    -4803808547255987458,
    -4770541604851511408,
    -4755786410560617033,
    -4653615893907202049,
    -4478736262549361059,
    -4306829139041234399,
    -4279216022012262472,
    -4192880338322568019,
    -4140416341754949840,
    -4002480899432492493,
    -3742920381661642450,
    -3740688946153302497,
    -3648965634801806860,
    -3468736362917677804,
    -3240342581500645782,
    -3203461052233479774,
    -3188756536805775746,
    -3175534537578382033,
    -3166235972286344185,
    -3111144504619311540,
    -3066585783639725785,
    -3056995598523672802,
    -2992816916116702016,
    -2986598439857071985,
    -2922409316302898713,
    -2901001809858326088,
    -2858452058372652657,
    -2826708484069621147,
    -2662956809214832758,
    -2653387738323233671,
    -2614848746731336897,
    -2561263686207170998,
    -2404562278962523548,
    -2404047898916485111,
    -2262286998071839118,
    -2242498560938898525,
    -2160176029458502177,
    -2072544378637865419,
    -1954655228122758087,
    -1782046196239035218,
    -1759795123698967886,
    -1750843003917527069,
    -1740153601367017929,
    -1592846799929684199,
    -1523465679562476684,
    -1505376806500647922,
    -1365681545539368470,
    -1336869408139887664,
    -1306865867554004770,
    -1303870168846898494,
    -1302957414581809033,
    -1169244505762834097,
    -1117771509786810887,
    -1000285923999366286,
    -977982561970768857,
    -909898093787164960,
    -782678436058281858,
    -449103398187555764,
    -389089412801867193,
    -377858537339604568,
    -350251637324427270,
    -315244910312855152,
    -151411256775773941,
    -141491101772582537,
    258546040704437327,
    403953861178593646,
    424547336101401673,
    442622999887944032,
    468695104951766377,
    509856259021940995,
    655059218690366573,
    840447456683045473,
    901698715864829018,
    942017474227255814,
    959421676899607792,
    1062123306922574183,
    1113974050375797352,
    1187550409515712842,
    1268893485366949549,
    1346540794543355186,
    1390674426398237791,
    1451359797735600888,
    1692335382724948275,
    1712868377592498463,
    1955235203570402306,
    1958734486838769788,
    1993985635909401007,
    2001123998283124912,
    2260143429692303010,
    2337416352229717633,
    2427380162772458209,
    2453910321092937254,
    2534746561067210107,
    2538568278643609627,
    2562936524635414141,
    2570353368425398533,
    2587544216927744961,
    2616523462074975962,
    2663280867033492967,
    2709050911413649226,
    2774625561457673521,
    2933838985882381251,
    2958293649439914938,
    2966701923321584899,
    3019911450441517882,
    3025451642978035157,
    3038048472611003357,
    3196549143782810422,
    3362592858756222877,
    3400614296296411680,
    3410607184992445741,
    3581096685468950840,
    3728414016101005629,
    3763080989213392264,
    3835973736681554299,
    3873960702544471546,
    3897176148818033906,
    3950208429840561275,
    4018617900742738998,
    4019303786617472180,
    4082561844170176454,
    4151639342333320211,
    4264880200200117452,
    4347910330932310459,
    4459638966225014230,
    4499035961557951230,
    4565270686668747294,
    4568397857501103489,
    4608675136892280590,
    4661348226989022374,
    4699299958812007250,
    4714036630911511306,
    4823165442025632533,
    4975460017277921112,
    4994496539339716938,
    5128275419372608034,
    5169146583571594538,
    5235094280214327175,
    5301757577695561155,
    5333126228484488401,
    5333773979046160403,
    5427170809583273887,
    5493563300814866402,
    5692998352156494932,
    5731779201691926731,
    5787422130295302958,
    5831814288992236774,
    5961644806904513581,
    6059204889195104411,
    6240465916934425470,
    6257474513515810341,
    6403965610528543392,
    6404237864193490179,
    6477993382853714906,
    6496834997774379646,
    6572552636058617243,
    6579336130424950645,
    6652244528876430950,
    6690493412484614695,
    6781012523694508817,
    6811521991764647140,
    6909860532231981817,
    6918046911334344647,
    6926540123153948470,
    6952843471066566661,
    6962750646577020133,
    6974385937177108071,
    7061287783591264150,
    7172213414777029478,
    7235280396329427624,
    7267656635003448811,
    7335962383781247087,
    7347286490828176254,
    7416643355177767134,
    7428357224966947601,
    7546148470936385857,
    7637192338914524118,
    7676478327801528289,
    7685047814542221766,
    7765139713679447583,
    7953161867420400402,
    7980427059310979987,
    7980845549166774262,
    8099046699285212505,
    8263815020182005589,
    8308096007075488501,
    8352253849850478228,
    8386132222141447885,
    8392050385495212231,
    8458264015709058476,
    8514327725551229135,
    8655752744440006404,
    8674092602322295441,
    8726631422100697046,
    8765274004819445634,
    8925705420633647239,
    9064439960759702216
  ],
  "schema": "backup/schema/cluster/d36c1432-df14-4b01-8329-d868fbb0ffae/task_3e4fb0c8-2e07-4bbd-b3a3-ab8d498d90cd_tag_sm_20231212164821UTC_schema.tar.gz",
  "index": [
    {
      "keyspace": "system_distributed_everywhere",
      "table": "cdc_generation_descriptions_v2",
      "version": "234d2227dd633d37ac5fc013e2ea9e6e",
      "files": [
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1a62_2c8eo2ugpkcdsb7w8k-big-TOC.txt",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1a62_2sy9c2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 43645
    },
    {
      "keyspace": "system_distributed",
      "table": "cdc_streams_descriptions_v2",
      "version": "0bf73fd765b236b085e5658131d5df36",
      "files": [
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-CompressionInfo.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Data.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Digest.crc32",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Filter.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Index.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Scylla.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Statistics.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-Summary.db",
        "me-3gbu_1a62_5grc02iffh61dprmyc-big-TOC.txt",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-CompressionInfo.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Data.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Digest.crc32",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Filter.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Index.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Scylla.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Statistics.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-Summary.db",
        "me-3gbu_1aol_3m3n42iffh61dprmyc-big-TOC.txt"
      ],
      "size": 42486
    },
    {
      "keyspace": "system_schema",
      "table": "columns",
      "version": "24101c25a2ae3af787c1b40ee1aca33f",
      "files": [
        "mc-6-big-CompressionInfo.db",
        "mc-6-big-Data.db",
        "mc-6-big-Digest.crc32",
        "mc-6-big-Filter.db",
        "mc-6-big-Index.db",
        "mc-6-big-Scylla.db",
        "mc-6-big-Statistics.db",
        "mc-6-big-Summary.db",
        "mc-6-big-TOC.txt"
      ],
      "size": 19817
    },
    {
      "keyspace": "system_schema",
      "table": "tables",
      "version": "afddfb9dbc1e30688056eed6c302ba09",
      "files": [
        "mc-6-big-CompressionInfo.db",
        "mc-6-big-Data.db",
        "mc-6-big-Digest.crc32",
        "mc-6-big-Filter.db",
        "mc-6-big-Index.db",
        "mc-6-big-Scylla.db",
        "mc-6-big-Statistics.db",
        "mc-6-big-Summary.db",
        "mc-6-big-TOC.txt"
      ],
      "size": 14935
    },
    {
      "keyspace": "system_distributed",
      "table": "cdc_generation_timestamps",
      "version": "fdf455c4cfec3e009719d7a45436c89d",
      "files": [
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-CompressionInfo.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Data.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Digest.crc32",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Filter.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Index.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Scylla.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Statistics.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-Summary.db",
        "me-3gbu_1a62_5w6w02iffh61dprmyc-big-TOC.txt",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-CompressionInfo.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Data.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Digest.crc32",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Filter.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Index.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Scylla.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Statistics.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-Summary.db",
        "me-3gbu_1aol_3l8s02iffh61dprmyc-big-TOC.txt"
      ],
      "size": 10778
    },
    {
      "keyspace": "system_schema",
      "table": "scylla_tables",
      "version": "5d912ff1f7593665b2c88042ab5103dd",
      "files": [
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_4u1fk2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 8008
    },
    {
      "keyspace": "system_schema",
      "table": "dropped_columns",
      "version": "5e7583b5f3f43af19a39b7e1d6f5f11f",
      "files": [
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_4vyvk2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 6185
    },
    {
      "keyspace": "system_schema",
      "table": "view_virtual_columns",
      "version": "08843b6345dc3be29798a0418295cfaa",
      "files": [
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_5dbvk2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 6228
    },
    {
      "keyspace": "system_schema",
      "table": "views",
      "version": "9786ac1cdd583201a7cdad556410c985",
      "files": [
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_5befk2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 6971
    },
    {
      "keyspace": "system_schema",
      "table": "computed_columns",
      "version": "cc7c7069374033c192a4c3de78dbd2c4",
      "files": [
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_5999s2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 6021
    },
    {
      "keyspace": "system_schema",
      "table": "keyspaces",
      "version": "abac5682dea631c5b535b3d6cffd0fb6",
      "files": [
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_51jhs2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5929
    },
    {
      "keyspace": "system_schema",
      "table": "functions",
      "version": "96489b7980be3e14a70166a0b9159450",
      "files": [
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_53gxs2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5785
    },
    {
      "keyspace": "system_schema",
      "table": "aggregates",
      "version": "924c55872e3a345bb10c12f37c1ba895",
      "files": [
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_556o02ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5688
    },
    {
      "keyspace": "system_schema",
      "table": "types",
      "version": "5a8b1ca866023f77a0459273d308917a",
      "files": [
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_4zm1s2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5580
    },
    {
      "keyspace": "system_schema",
      "table": "indexes",
      "version": "0feb57ac311f382fba6d9024d305702f",
      "files": [
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_57bts2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5565
    },
    {
      "keyspace": "system_schema",
      "table": "scylla_aggregates",
      "version": "08d8cb2892023371a968ad926e0fdc37",
      "files": [
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_4r94g2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5534
    },
    {
      "keyspace": "system_schema",
      "table": "triggers",
      "version": "4df70b666b05325195a132b54005fd48",
      "files": [
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_4xwbk2ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5519
    },
    {
      "keyspace": "system_schema",
      "table": "scylla_keyspaces",
      "version": "fa0ea2bd608f3e749b1eb84b46b33adf",
      "files": [
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-CompressionInfo.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Data.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Digest.crc32",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Filter.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Index.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Scylla.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Statistics.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-Summary.db",
        "me-3gbu_1aol_4sjf42ugpkcdsb7w8k-big-TOC.txt"
      ],
      "size": 5499
    },
    {
      "keyspace": "system_traces",
      "table": "node_slow_log_time_idx",
      "version": "f9706768aa1e3d879e5c51a3927c2870",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_traces",
      "table": "node_slow_log",
      "version": "bfcc4e625b633aa1a1c36f5e47f3325c",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_traces",
      "table": "events",
      "version": "8826e8e9e16a372887533bc1fc713c25",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_auth",
      "table": "role_attributes",
      "version": "6b8c7359a84333f2a1d85dc6a187436f",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_traces",
      "table": "sessions_time_idx",
      "version": "0ebf001cc1d136939a63c3d96ac53318",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_distributed",
      "table": "view_build_status",
      "version": "5582b59f8e4e35e1b9133acada51eb04",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_distributed",
      "table": "service_levels",
      "version": "b8c556bd212d37ad9484690c73a5994b",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_traces",
      "table": "sessions",
      "version": "c5e99f1686773914b17e960613512345",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_auth",
      "table": "roles",
      "version": "5bc52802de2535edaeab188eecebb090",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_auth",
      "table": "role_permissions",
      "version": "3afbe79f219431a7add7f5ab90d8ec9c",
      "files": [],
      "size": 0
    },
    {
      "keyspace": "system_auth",
      "table": "role_members",
      "version": "0ecdaa87f8fb3e6088d174fb36fe5c0d",
      "files": [],
      "size": 0
    }
  ]
}

@Michal-Leszczynski
Copy link
Collaborator Author

This PR managed to get rid of the following error in ansible script:

fatal: [2fc8a10793f1]: FAILED! => {"changed": true, "cmd": "scylla-manager-agent download-files -L s3:backuptest-smoke -n a35fd50a-33db-487d-b95d-7455c8602ddf -T sm_20240322140450UTC --dump-manifest | jq -r '.index[] | [.keyspace,.table] | join(\".\")'\n", "delta": "0:00:00.077102", "end": "2024-03-22 14:10:23.990879", "msg": "non-zero return code", "rc": 5, "start": "2024-03-22 14:10:23.913777", "stderr": "jq: error (at <stdin>:265): Cannot iterate over null (null)", "stderr_lines": ["jq: error (at <stdin>:265): Cannot iterate over null (null)"], "stdout": "", "stdout_lines": []}

but I failed to validate if the scripts works correctly on our docker env setup.

@Michal-Leszczynski Michal-Leszczynski marked this pull request as ready for review March 22, 2024 15:24
@Michal-Leszczynski
Copy link
Collaborator Author

@karol-kokoszka could you take a look at this PR?

karol-kokoszka
karol-kokoszka approved these changes Mar 27, 2024
View reviewed changes
Copy link
Collaborator

@karol-kokoszka karol-kokoszka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Michal-Leszczynski Michal-Leszczynski merged commit 5cebaf5 into master Mar 27, 2024
19 of 21 checks passed
@Michal-Leszczynski Michal-Leszczynski deleted the ml/fix-dump-manifest branch March 27, 2024 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karol-kokoszka karol-kokoszka karol-kokoszka approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

scylla-manager-agent --dump-manifest doesn't contain index files
2 participants
@Michal-Leszczynski @karol-kokoszka