Allow list broken with 1.20.10 update.

[IanSmith232]

Not sure what is happening exactly, but with the 1.20.10 update, I can't join my server. It says "you are not invited" or something.

Here is my environment variables. I removed the ALLOW_LIST_USERS all together, and I still get rejected from the server.

 "environmentVariables": [
                        {
                            "name": "LEVEL_NAME",
                            "value": "bedrock-level-2"
                        },
                        {
                            "name": "DEBUG",
                            "value": "true"
                        },
                        {
                            "name": "ALLOW_CHEATS",
                            "value": "false"
                        },
                        {
                            "name": "MEMBERS",
                            "value": "xxx,yyy,zzz"
                        },
                        {
                            "name": "EULA",
                            "value": "true"
                        },
                        {
                            "name": "DIFFICULTY",
                            "value": "easy"
                        },
                        {
                            "name": "GAMEMODE",
                            "value": "survival"
                        },
                        {
                            "name": "OVERRIDE_SERVER_PROPERTIES",
                            "value": "true"
                        },
                        {
                            "name": "TZ",
                            "value": "Europe/Amsterdam"
                        },
                        {
                            "name": "OPS",
                            "value": "xxx"
                        },
                        {
                            "name": "VERSION",
                            "value": "LATEST"
                        },
                        {
                            "name": "MAX_PLAYERS",
                            "value": "10"
                        }
                    ],

[itzg]

I'm still not sure what is happening. The latest image now generates the allowlist.json file exactly as the allowlist add ... command does; however, I get the same "can't join the party". The latest image does remove the allowlist file when ALLOW_LIST_USERS is empty/absent, so at least you can get the server back into a joinable state.

[itzg]

...I'm going to mark this as "caused by upstream" for now, because something seems to be weird on their part. It's like the allow list needs to only be loaded into the world data via the command.

[IanSmith232]

Verified that the workaround (deleting allow list) works. Thank you for looking into this!!!!!

[SvenLudwig202]

I'm still not sure what is happening. The latest image now generates the allowlist.json file exactly as the allowlist add ... command does; however, I get the same "can't join the party". The latest image does remove the allowlist file when ALLOW_LIST_USERS is empty/absent, so at least you can get the server back into a joinable state.

With those latest changes (commit #337) it seems to no longer be possible to get a "sideloaded" allowlist.json to work, as it ends up being deleted when you don't have the ALLOW_LIST_USERS filled and overwritten when you set said variable.
Is this intended?

[itzg]

@SvenLudwig202 can you help figure out why the presence of an allowlist.json file prevents joining the server? When you side load the file, is there anything in particular you put in yours for the listed users to be able to access th server?

I'll change the code later to at least only delete the file when the variable is an empty string.

[SvenLudwig202]

To be fully honest I don't think I can bring much new info to the table.

I noticed the bedrock server not starting with an error message pointing out that it wasn't able to delete the allowlist.json.
I had ran my server with setting the ALLOW_LIST: "true" and then splicing in the allowlist.json via the volumes in the compose file for quite some time with no problems. Though to be fully honest, I haven't tried connecting to the server since the latest
1.20.11 release.
This allowlist.json looked nothing special, see below:
[{"ignoresPlayerLimit":false,"name":"Iamaname","xuid":"123456789123456789"},{"ignoresPlayerLimit":false,"name":"Alsoaname","xuid":"987654321987654321"}]

I've now tried setting up using the ALLOW_LIST_USERS by entering the required usernames separated via comma and I can see that a allowlist.json is being generated (missing the xuid portion at this point). From what I understand the bds should read this and resolve the names, but this doesn't seem to be happening. The server isn't putting out any info to the log regarding this though. I've tried to force a reload via "allowlist reload" which is being executed and written into the server's log.
Yet I'm in fact not able to join the server as Ian pointed out in his original message ("you're not invited ..").

If you see any chance of me helping out here feel free to let me know. I'm more than happy to try out a view things.

[itzg]

Though to be fully honest, I haven't tried connecting to the server since the latest
1.20.11 release.

Thanks @SvenLudwig202 that's a very helpful data point. It seems like it is something about that version.

I just pushed a new image that will only remove the allowlist/whitelist files when the ALLOW_LIST_USERS variable set to an empty string.

With that pulled it should now be safe to put back the file you were using FWIW.

[SvenLudwig202]

I can confirm that with the new image my setup does indeed work again and I can connect to my server.

It does seem like the current bds version isn't resolving the allowlist.json for whatever reason. If the xuid are present, like in my case, it does seem to work flawless.

For reference, this is the compose file I'm running right now.

version: '3.4'
services:
bds:
image: itzg/minecraft-bedrock-server
environment:
UID: "1002"
GID: "1002"
EULA: "TRUE"
GAMEMODE: survival
DIFFICULTY: normal
SERVER_NAME: "XXXX Bedrock"
MAX_PLAYERS: "4"
LEVEL_NAME: "Bedrock2"
ALLOW_LIST: "true"
ports:
- 9132:19132/udp
volumes:
- ./data:/data
- ./extra/allowlist.json:/data/allowlist.json
stdin_open: true
tty: true
restart: always

(can't seem to get the spacing to work in either quote or code mode .. please just imagine the correct indentation being there)

[itzg]

Ah, ok, so with name and xuid present it works? I wish there was an API on the bedrock/Xbox side to solve names to xuid. I could add that to the script, if so.

Their own docs say xuid is optional, so it still seems like they broke something. Maybe we wait to see with next version.

[ewolkowicz]

@itzg I updated a bit ealier today (client 1.20.10, server 1.20.11). What @SvenLudwig202 identified was correct but I found another issue. You have to explicitly delete whitelist.json, otherwise you'll still get the "not invited" error. I'm guessing they removed some references to it but still parse it at some point during loading. This is definitely a bug upstream. I haven't tested if adding the same params to the whitelist.json file fixes that as well, but just removing it, switching to allowlist and doing both username and xuid works. Is it worth submitting a PR for a workaround or just waiting to see what Mojang does? (or is there already a bug filed with them?)

[itzg]

Good find! I'd be good with a PR to have the startup script make sure whitelist.json isn't present. I did notice a deprecation log when that file was present.

[itzg]

Latest image now tweaks the logic as @ewolkowicz found and it seems to be working for me now.

[xavierly]

I notice a strange behavior. With ALLOW_LIST_USERS and ALLOW_LIST=TRUE set, previously the allowed users see "Not Invited" error. After removing allowlist.json and restarting the container, anybody is now allowed to connect with the same ALLOW_LIST_USERS and ALLOW_LIST.

[itzg]

The bedrock software is acting like it saves its own list within the world data perhaps. This is all strange and hopefully gets better with their next release.

I still haven't looked in their bug tracker for anything about this. Has anyone else looked?

[AbeHaruhiko]

I think this bug fix may be related to the behavior.
https://bugs.mojang.com/browse/BDS-18133

[puddlewitt]

It looks like, understandably, I can't add the xuid to the ALLOW_LIST_USERS environment variable. For my scenario I've manually added the xuid to allowlist.json for the time being, which works.

[itzg]

It looks like, understandably, I can't add the xuid to the ALLOW_LIST_USERS environment variable. For my scenario I've manually added the xuid to allowlist.json for the time being, which works.

Correct. Instead you need to add your gamer tag to that list.

[arberg]

I can confirm the following in Bedrock version 1.20.12.01.

allowList.json does not work without xuid (Player not invited), but does work when using xuid's in allowList.json.

The following docker-compose (thank you @SvenLudwig202 above) works. It mounts allowlist.json in docker and I have already added my xuid's manually to the file. I have removed docker-compose environment ALLOW_LIST. By working I mean I can start the docker and login with my users without further ado.

version: '3.2'
# UID 99 is nobody
services:
  mc:
    container_name: minecraft
    image: itzg/minecraft-bedrock-server:latest
    restart: always
    volumes:
    - ./data:/data
    - ./extra/allowlist.json:/data/allowlist.json
    - /etc/localtime:/etc/localtime:ro
    environment:
      VERSION: 1.20.12.01
      ONLINE_MODE: "true"
      GAMEMODE: "creative"
      DIFFICULTY: "normal"
      # ALLOW_LIST_USERS: disabled this because it does not support xuid's
      ALLOW_LIST: "true"
      GUI: "false"
      EULA: "true"
      UID: 99
      GID: 100
    ports:
      - 19132:19132/udp
    # With 'tty: true' its possible to attach and run commands: docker attach CONTAINER_NAME_OR_ID > op YOUR_XBOX_USERNAME
    stdin_open: true
    tty: true
    ```
      
extra/allowlist.json (sample, i'm sure the ignoresPlayerLimit is not needed, but I haven't tested it)

[{
"ignoresPlayerLimit": false,
"name": "MY_TAG",
"xuid": "1234567891234567"
}]

Note that the extra/allowlist.json file above must exists, or an empty directory is created and the container cannot start. The allowlist.json can initially contain '[]'.

------

Without allowlist.json (with xuids):

If I use 'docker attach minecraft' and run 

allowlist add MY_NAME
allowlist list

then I can login in Minecraft with MY_NAME user.
If I then in the console run 'allowlist list' again (or just check log) I can see my xuid in the allowlist.json printed. 

---
Without allowlist.json (with xuids):

If I as above run the allowlist commands, but then restart the docker container (before logging in) then I cannot login (Player not invited). So the part of the allowlist command effect that allows lookup of xuid's is kept in memory and only works for that session. However the good news is that when a user logs in (before server is restarted) Minecraft writes the xuid to the allowlist.json file (which is preserved outside the container in this docker-compose script) and thus user can login after server restart.

---

It seems pretty obvious that it is some kind of bug in the new Minecraft (or new extra security), because allowlist command actually writes gametags without xuid in allowlist.json, and updates it with xuid once a player logs in the first time.

---

I wonder if its possible to write a script that runs on docker start (such as in the docker container post minecraft start) that runs the allowlist commands based on the environment ALLOW_LIST. But maybe its not worth it if its fixed soon by the Minecraft Team.

[xavierly]

I found this site that retrieve xuid from the gametag: https://www.cxkes.me/xbox/xuid

[itzg]

Awesome find @xavierly !

Anybody want to PR that into the startup script?

I'm also trying to decide if I'll add a subcommand to https://github.com/itzg/mc-image-helper to allow more robust add/replace behaviors. A PR to add allowlist management there would be great too.

[fishscene]

#336 (comment)
This works perfectly, and using the console to add "allowlist add (xboxUsername)" works perfectly- it even grabs the XUID. I don't have to know or touch XUID's to add or remove folks to the allowlist.

Instead of manually manipulating the file, would it be better just to use the tools provided?

For example:

1. On container startup parse through the output of "allowlist list", purge all entries.
2. Add entries based on the "ALLOW_LIST_USERS" variable.

[arberg]

Hi fishscene, According to my tests above, the 'allowlist add X' HAD to be executed in the running instance. That might not be easy to do, programmatically in the docker. Also in my experience it can only grab the xuid's when my users actually logged in, which was a bit of a pain.

I think 1+2 is exactly what itzg already does in current version, he just didn't have access to the XUID's.

[itzg]

The container startup only operates on the files directly and that's all it can do. Once the bedrock process is executed the container startup script is completely out of the picture. The crux of the original problem was that the bedrock software was not honoring the allowlist file when the whitelist file is present.

It feels like this github issue has diverged in several directions, so I'm quite confused how to proceed. The original problem was fixed, so I'm inclined to close this issue and I'd appreciate people open a new issue for each new specific feature/issue identified.

[DrFrankensteinUK]

Hey All

I am completely lost - what is the actual solution in the end for this issue as I seem to be going around in circles.

[fishscene]

@itzg
The original problem "You are not invited to this server", stemming from the allowlist changes, has been fixed?
If so, can we just use:

ALLOW_LIST: 'true'
ALLOW_LIST_USERS: '[redacted],[redacted],[redacted],[redacted],[redacted],[redacted],[redacted]'

?
Or is there additional work, as I've written below?

@DrFrankensteinUK
Here is my exact Portainer config, with some redactions.
Basically, ENABLE the allow list, DISABLE the Allow_list_users.
When the docker container fires up, attach to the console and manually add folks to the allow list by NAME.
I did not have to deal with XUID's at all.
For example:
allowlist add sketchyburger

Upon checking the allowlist, both the username and XUID were present without anyone having to log in or having to manually look up the XUID.

#Allowlist is broken, so to allow players, attach to docker console (Portainer paper clip icon) and run "allowlist add [redacted]" (Case-sensitive)
#Every time docker container is updated or re-created (Not start/stopped), must re-add players.
#allowlist add [redacted]
#allowlist add [redacted]
#allowlist add [redacted]
#allowlist add [redacted]
#allowlist add [redacted]
#allowlist add [redacted]
#allowlist list ## Checks list


version: "3"

volumes:
  data:
  
services:
  minecraft:
    container_name: mc-couraud
    hostname: mc-couraud
    image: itzg/minecraft-bedrock-server:latest
    ports:
      - 19132:19132/udp
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - data:/data
    tty: true # With 'tty: true' its possible to attach and run commands: docker attach CONTAINER_NAME_OR_ID > op YOUR_XBOX_USERNAME
    stdin_open: true ## Allows attaching to docker container.
    environment:
      TZ: 'America/Los_Angeles'
      ALLOW_CHEATS: 'true'
      DIFFICULTY: 'normal'
      EULA: 'TRUE'
      GUI: 'FALSE'
      GAMEMODE: 'survival'
      SERVER_NAME: 'minecraft'
      SERVER_PORT: '19132'
      ALLOW_LIST: 'true'
      OPS: "[redacted long number]"
      MAX_Players: "10"
      #ALLOW_LIST_USERS: '[redacted],[redacted],[redacted],[redacted],[redacted],[redacted],[redacted]'
    restart: 'always'

[itzg]

#346 just got merged, which might clear up the remaining issues here. My testing was down with no prior data, so I had missed the case where there would have been a left over whitelist file.

[itzg]

There's some kind of inconsistency with that bug in Bedrock server: I enhanced the image to remove any left over white-list property in server.properties yet it doesn't allow me to join. However...one of the times I tested it did. It is either sporadic or extremely particular.

I'm running out of ideas and at wits end -- we may just have to see if the next bedrock version fixes all this.

[hymerman]

Mounting /data to a docker volume, and inspecting the contents of that volume, my allowlist.json file is being created on container startup, set to an old list of players (I believe the same list I used to use, before a recent update), none of which have XUIDs. "allow-list" is set to true in server.properties. All players get rejected when they try to join. Attaching to the container, "allowlist list" shows an empty list (which is inconsistent with what allowlist.json says). Doing "allowlist add ABC" then resets the contents of allowlist.json to contain just that player - all updates from then on get reflected in allowlist.json (and allowlist list), including setting XUIDs when those players join. This is all lost on container restart - it gets reset back to that old list of player names without XUIDs.

Binding a local allowlist.json to /data/allowlist.json does nothing - they seem completely disconnected from each other.

All the whitelist/allowlist environment variables in the docker compose config appear to do nothing now too. allow-list is ALWAYS set to true, and the allowlist.json file ALWAYS has the same contents (the old list of player names without XUIDs).

I wonder if allow-list and allowlist.json are now overwritten on server startup, based on something saved elsewhere in the container? If so, that's a server bug, and it doesn't sound like the one linked above :(

[itzg]

So it sounds like the startup script management of allowlist.json just needs to be removed for now, perhaps with a startup warning in the logs, since it seems to be counterproductive to command-driven updates.

Any suggestions otherwise are very welcome.

Honestly I really need others to contribute PRs for this. Personally I only use Java edition along with Geyser for bedrock compatibility. So I have no personal interest in fighting the disappointing quality of the Bedrock server software provided by Mojang. (The Bedrock software has been error prone for years, so this a long brewing frustration for me)

[Laptop765]

Binding a local allowlist.json to /data/allowlist.json does nothing - they seem completely disconnected from each other.

Ah! It's likely this is because of the behavior when only binding a single file instead of a directory. If you bind allowlist.json directly the script looks like it overwrites it: jq -c -n --arg users "$allowListUsers" '$users | split(",") | map({"ignoresPlayerLimit":false,"name": .})' > "allowlist.json". Assuming that > does create a new file rather than updating in place this breaks the bind so when the file is updated it doesn't propagate outside of the container.

[Laptop765]

Honestly I really need others to contribute PRs for this.

@itzg Are you open to contributions being in another language, e.g. Python? I thought about contributing auto-install addons functionality but my Bash skills are pretty weak.

[itzg]

Normally I would say no since all of my existing tooling is bash, Go, or Java based and wouldn't want to grow the image size in the direction of Python. ...however, I'm frustrated enough that I'm totally open to that 😀

With that said, can you first summarize what you propose contributing?

The technical solutions are easy for me to implement -- what is difficult is identifying what exactly would convince the bedrock server to accept an externally crafted allowlist file and how to resolve XUIDs without using an unofficial web service.

[itzg]

...with those answers, I would really prefer to see the implementation added into

https://github.com/itzg/mc-image-helper/blob/master/src/main/java/me/itzg/helpers/users/ManageUsersCommand.java

I only have a local branch queued up to introduce mc-image-helper, so can push that sooner rather than later if needed.

[Laptop765]

I could definitely do Java over Bash 🙂 What's the relation between this repo and mc-image-helper?

[itzg]

@Laptop765 just pushed #351 which brings in the new usage of mc-image-helper

[Laptop765]

I'm wondering if allowlist add performs an XUID lookup while the server is running and that's why it works when plain usernames in the JSON file don't. @itzg Is there a mechanism for running commands on the server after startup? We could maybe fix this that way - boot the server, get the current allow list, calculate the diff (adds/removes) from the environment variable and then run server console commands to do the adds/removes.

[itzg]

Is there a mechanism for running commands on the server after startup?

Not as currently designed. The script execs the bedrock executable which swaps out the script from the running process:

docker-minecraft-bedrock-server/bedrock-entry.sh

Line 194 in 2888c55

exec ./bedrock_server-${VERSION}

I really wish Microsoft provided an XUID lookup API like the Java/Mojang user lookup:

https://github.com/itzg/mc-image-helper/blob/6aeedd00fe4a3af943c886f2b042a3234c80fe8c/src/main/java/me/itzg/helpers/users/ManageUsersCommand.java#L253-L256

Last I looked it was behind an API key mechanism that required an extensive application process.

[itzg]

It is as tedious as I remember and not well suited for a distributed tool: https://stackoverflow.com/a/65693218

[Laptop765]

It is as tedious as I remember and not well suited for a distributed tool: https://stackoverflow.com/a/65693218

I figured that was probably the case when I wasn't able to find an answer with a quick search 😢

If I'm reading the code correctly setting neither ALLOW_LIST_USERS nor WHITE_LIST_USERS should entirely skip the allowlist logic?

If that's the case then the current workaround should be (currently untested):
1. Remove both of those variables from the container
2. Bind an allowlist.json file through Docker
3. Use allowlist add {username} directly on the server to manage the list

However this depends on whether the server updates the file in-place instead of creating a new one when using allowlist add. See my comment above for more details on why this matters. If anyone has a chance to test this before I do please report back 🙂 I'm not sure when I'll have the time.

EDIT: I decided to spend a few minutes with this. I think it's actually just very broken. It seems that whenever the list is loaded anything missing an XUID is removed. So if a player is allowlist added and they join before the list is reloaded through allowlist reload or the server is restarted, they should be fine. This issue also has more info.

[itzg]

Nice research finding those issues. Those look like the culprits. So, I guess we're kinda stuck until those get addressed?

[Laptop765]

Nice research finding those issues. Those look like the culprits. So, I guess we're kinda stuck until those get addressed?

Sadly I think so 😔

[rjbradleyling]

Personally I only use Java edition along with Geyser for bedrock compatibility. So I have no personal interest in fighting the disappointing quality of the Bedrock server software provided by Mojang. (The Bedrock software has been error prone for years, so this a long brewing frustration for me)

I too am becoming frustrated with Bedrock's bugginess and would like to move to Java, but I have children that only have access to iPads for gaming and are thus locked to the Bedrock edition. Your mention of Geyser interests me. Can you point me to a good starting resource for implementing Geyser to work with your Java Docker image?

[itzg]

@rjbradleyling here's the example I use to connect with an iPhone https://github.com/itzg/docker-minecraft-server/blob/master/examples/geyser/docker-compose.yml