Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo configuration script requires admin password #3329

Merged
merged 112 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
37f5fc4
Base changes
stephen-crawford Sep 13, 2023
0e06014
Merge branch 'opensearch-project:main' into adminConfigFile
stephen-crawford Sep 13, 2023
ec8bf8c
Merge branch 'opensearch-project:main' into adminConfigFile
stephen-crawford Sep 13, 2023
66b81ef
state of the world
stephen-crawford Sep 13, 2023
4079fe8
swap to separate fie
stephen-crawford Sep 13, 2023
d0b26ed
reset sec plugin.java
stephen-crawford Sep 13, 2023
a2dd667
Reset config constants
stephen-crawford Sep 13, 2023
edb4117
Merge branch 'opensearch-project:main' into adminConfigFile
stephen-crawford Sep 19, 2023
bba2e5f
update hash function
stephen-crawford Sep 20, 2023
4c5b6f0
Merge branch 'opensearch-project:main' into adminConfigFile
stephen-crawford Sep 20, 2023
0a1403f
remove space
stephen-crawford Sep 20, 2023
18c63f0
Fix plugin install
stephen-crawford Sep 20, 2023
e1c23cc
move dir out of config
stephen-crawford Sep 20, 2023
1375996
move file
stephen-crawford Sep 20, 2023
fcabdd7
try path
stephen-crawford Sep 20, 2023
c42e2a1
try path
stephen-crawford Sep 20, 2023
96c06a6
List files
stephen-crawford Sep 20, 2023
bbb9579
View dir
stephen-crawford Sep 20, 2023
e00a64b
fix paths
stephen-crawford Sep 20, 2023
1261c3f
Print dir
stephen-crawford Sep 20, 2023
e7c23d6
Try again
stephen-crawford Sep 20, 2023
a44f073
Retry
stephen-crawford Sep 20, 2023
0a349df
print out sec dirr
stephen-crawford Sep 20, 2023
d29e805
print out sec dirr
stephen-crawford Sep 20, 2023
976c8ab
print out sec dir in config
stephen-crawford Sep 20, 2023
ca8b48b
update order
stephen-crawford Sep 20, 2023
c9f245d
update order
stephen-crawford Sep 20, 2023
e939ab6
Try head instead
stephen-crawford Sep 20, 2023
6de3f67
remove quotes
stephen-crawford Sep 20, 2023
542a66f
try env var
stephen-crawford Sep 20, 2023
5dfbf9f
try env var
stephen-crawford Sep 20, 2023
c2bbc93
cofirm correct file
stephen-crawford Sep 20, 2023
922cd06
update
stephen-crawford Sep 20, 2023
048e650
update
stephen-crawford Sep 20, 2023
54ee921
move password population
stephen-crawford Sep 21, 2023
2376623
add prints
stephen-crawford Sep 21, 2023
9d63dcd
try env var
stephen-crawford Sep 21, 2023
a8e5c2d
list dirs
stephen-crawford Sep 21, 2023
fa47990
list dirs
stephen-crawford Sep 21, 2023
c820e78
checking setting
stephen-crawford Sep 21, 2023
1f60bb1
try again
stephen-crawford Sep 21, 2023
18f5715
check
stephen-crawford Sep 21, 2023
48856d2
please work
stephen-crawford Sep 21, 2023
95e6e8f
Change if
stephen-crawford Sep 21, 2023
59b25e1
Try windows
stephen-crawford Sep 21, 2023
6cf04b2
try assignements
stephen-crawford Sep 21, 2023
e4467b8
test
stephen-crawford Sep 21, 2023
d28bceb
Try again
stephen-crawford Sep 21, 2023
1f16c5d
retry sed
stephen-crawford Sep 21, 2023
d308fb4
test with temp file
stephen-crawford Sep 22, 2023
3050899
test
stephen-crawford Sep 22, 2023
005d70a
try sed again
stephen-crawford Sep 22, 2023
b5e7d4e
try sed again
stephen-crawford Sep 22, 2023
f13dbdd
try again
stephen-crawford Sep 22, 2023
a40102c
Escape $
stephen-crawford Sep 22, 2023
011509f
Add slash
stephen-crawford Sep 22, 2023
72440cf
try awk
stephen-crawford Sep 22, 2023
a5166d9
test output
stephen-crawford Sep 22, 2023
66aa2dc
test windows
stephen-crawford Sep 22, 2023
e962632
test
stephen-crawford Sep 22, 2023
0e904e8
move back
stephen-crawford Sep 22, 2023
31b63d9
Try with first back then forward slashes
stephen-crawford Sep 22, 2023
f6f4d60
try again
stephen-crawford Sep 22, 2023
c4658b9
prints
stephen-crawford Sep 22, 2023
f8b81e4
print dirs
stephen-crawford Sep 22, 2023
c8a1d88
test with env var
stephen-crawford Sep 22, 2023
1116ea0
Try set and get content
stephen-crawford Sep 22, 2023
d557de8
Try set and get content
stephen-crawford Sep 22, 2023
7e52f35
Try modifying setup
stephen-crawford Sep 22, 2023
57a6bac
Try modifying setup
stephen-crawford Sep 22, 2023
c14b65d
Fix pathes
stephen-crawford Sep 22, 2023
c8a605e
test
stephen-crawford Sep 22, 2023
8ae73e9
Try setting
stephen-crawford Sep 22, 2023
12b45f9
Fix set
stephen-crawford Sep 22, 2023
595a82b
Fix set
stephen-crawford Sep 22, 2023
0d58f65
Try again
stephen-crawford Sep 22, 2023
142cac0
escape quotes
stephen-crawford Sep 22, 2023
a849f15
remove quotes
stephen-crawford Sep 22, 2023
0081cfe
try coverted awk
stephen-crawford Sep 22, 2023
d3b2371
fix pattern
stephen-crawford Sep 22, 2023
264661e
Check for any number of spaces
stephen-crawford Sep 22, 2023
f69050a
Try 2 leading spaces looped line trimming
stephen-crawford Sep 22, 2023
b216245
try to update file different way
stephen-crawford Sep 25, 2023
294aba8
try another way
stephen-crawford Sep 25, 2023
023a580
Try setting locals
stephen-crawford Sep 25, 2023
1a6094c
remove one level of quotes
stephen-crawford Sep 25, 2023
a0012ca
try again
stephen-crawford Sep 25, 2023
79007c8
remove echos
stephen-crawford Sep 25, 2023
1eec0c6
Add deprecation warnings back to the top of hash tools
peternied Sep 26, 2023
6c093e3
Clean up changes to the linux shell script
peternied Sep 26, 2023
3839cde
Clean up some of the win batch file
peternied Sep 26, 2023
ac0a2a3
Pass admin password as a parameter in start cluster action
peternied Sep 26, 2023
a5956d1
Accept password from job input
peternied Sep 26, 2023
d6d71ec
Create the password files in the action
peternied Sep 26, 2023
016cf09
Restore original file population
peternied Sep 26, 2023
af5fd76
Make sure CURL fails if there is a 400+ error code response
peternied Sep 26, 2023
03e5f10
Debug password isn't defined well
peternied Sep 26, 2023
22b53d4
Set the RNG value to another env so it doesn't change after use
peternied Sep 26, 2023
0f8e4c9
Fiix incomplete echo prompt to set the password
peternied Sep 26, 2023
3e4e8b0
Restore to main as much as possible
peternied Sep 26, 2023
ae34718
Restore batch file as much as possible
peternied Sep 26, 2023
633c020
Shell script revert as much as possible
peternied Sep 26, 2023
3bba55a
Revert "Restore batch file as much as possible"
peternied Sep 26, 2023
6feb4c3
Fix mixed locations of the password file
peternied Sep 26, 2023
5147d23
Make sure to tail output from hasher to ignore deprecation message
peternied Sep 26, 2023
b70a2c0
Add debug log for config directory
peternied Sep 26, 2023
b94943f
Narrow in on the password file issue in windows
peternied Sep 26, 2023
91949ca
Use action to save file to the filesystem
peternied Sep 27, 2023
1581148
Use functional RNG on windows
peternied Sep 27, 2023
bef523e
Use fixed version of the password generation tool
peternied Sep 27, 2023
917d7af
Use full commit sha
peternied Sep 27, 2023
d2a6de2
Switch to published version of the name generator
peternied Sep 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/admin_password.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
testPassword
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion config/internal_users.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ _meta:
## Demo users

admin:
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
hash: "$2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG"
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
hash:
reserved: true
backend_roles:
- "admin"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -283,8 +283,8 @@ public void testDefaultConfig() throws Exception {
RestHelper rh = nonSslRestHelper();
Thread.sleep(10000);

Assert.assertEquals(HttpStatus.SC_OK, rh.executeGetRequest("", encodeBasicHeader("admin", "admin")).getStatusCode());
HttpResponse res = rh.executeGetRequest("/_cluster/health", encodeBasicHeader("admin", "admin"));
Assert.assertEquals(HttpStatus.SC_OK, rh.executeGetRequest("", encodeBasicHeader("admin", "testPassword")).getStatusCode());
HttpResponse res = rh.executeGetRequest("/_cluster/health", encodeBasicHeader("admin", "testPassword"));
Assert.assertEquals(res.getBody(), HttpStatus.SC_OK, res.getStatusCode());
}

Expand All @@ -300,14 +300,14 @@ public void testInvalidDefaultConfig() throws Exception {
Thread.sleep(10000);
Assert.assertEquals(
HttpStatus.SC_SERVICE_UNAVAILABLE,
rh.executeGetRequest("", encodeBasicHeader("admin", "admin")).getStatusCode()
rh.executeGetRequest("", encodeBasicHeader("admin", "testPassword")).getStatusCode()
);

ClusterHelper.updateDefaultDirectory(defaultInitDirectory);
restart(Settings.EMPTY, null, settings, false);
rh = nonSslRestHelper();
Thread.sleep(10000);
Assert.assertEquals(HttpStatus.SC_OK, rh.executeGetRequest("", encodeBasicHeader("admin", "admin")).getStatusCode());
Assert.assertEquals(HttpStatus.SC_OK, rh.executeGetRequest("", encodeBasicHeader("admin", "testPassword")).getStatusCode());
} finally {
ClusterHelper.resetSystemProperties();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ protected void setup(
Settings nodeOverride,
boolean initSecurityIndex
) throws Exception {
setup(initTransportClientSettings, dynamicSecuritySettings, nodeOverride, initSecurityIndex, ClusterConfiguration.DEFAULT);
Settings settings = Settings.builder().put(nodeOverride).put("plugins.security.bootstrap.admin.password", "testPassword").build();
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
setup(initTransportClientSettings, dynamicSecuritySettings, settings, initSecurityIndex, ClusterConfiguration.DEFAULT);
}

protected void restart(
Expand Down
62 changes: 62 additions & 0 deletions tools/install_demo_configuration.bat
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ cd %CUR%
echo Basedir: %BASE_DIR%

set "OPENSEARCH_CONF_FILE=%BASE_DIR%config\opensearch.yml"
set "INTERNAL_USERS_FILE"=%BASE_DIR%config\internal_users.yml"
set "ADMIN_PASSWORD_FILE"=%BASE_DIR%config\admin_password.txt"
set "OPENSEARCH_CONF_DIR=%BASE_DIR%config\"
set "OPENSEARCH_BIN_DIR=%BASE_DIR%bin\"
set "OPENSEARCH_PLUGINS_DIR=%BASE_DIR%plugins\"
Expand Down Expand Up @@ -319,6 +321,66 @@ echo plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_a
echo plugins.security.system_indices.enabled: true >> "%OPENSEARCH_CONF_FILE%"
echo plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*", ".opendistro-job-scheduler-lock"] >> "%OPENSEARCH_CONF_FILE%"


REM Initialize the variable
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
set "ADMIN_PASSWORD="

REM Read the content of admin_password.txt into the ADMIN_PASSWORD variable
for /f "usebackq" %%i in ("%ADMIN_PASSWORD_FILE%") do (
set "ADMIN_PASSWORD=%%i"
)

REM If ADMIN_PASSWORD is empty, check the environment variable as a fallback
if not defined ADMIN_PASSWORD (
if defined ENV_ADMIN_PASSWORD (
set "ADMIN_PASSWORD=!ENV_ADMIN_PASSWORD!"
) else (
echo Unable to find admin password for cluster, please run "set ENV_ADMIN_PASSWORD=<your_password>" or create a file {OPENSEARCH_ROOT}\admin_password.txt with a single line that contains the password followed by a newline.
exit /b 1
)
)


set "salt="
for /l %%i in (1,1,16) do (
set /a "rand=!random! %% 16"
set "salt=!salt!!rand!"
)

openssl passwd -bcrypt -salt !salt! "!ADMIN_PASSWORD!" > tmp_hash.txt
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved

set "HASHED_ADMIN_PASSWORD="
for /f %%a in (tmp_hash.txt) do (
set "HASHED_ADMIN_PASSWORD=%%a"
)

del tmp_hash.txt

for /f "tokens=1 delims=:" %%b in ('findstr /n "admin:" "%INTERNAL_USERS_FILE%"') do (
set "ADMIN_HASH_LINE=%%b"
)

(for /f "delims=" %%c in ('type "%INTERNAL_USERS_FILE%" ^| findstr /n "^"') do (
set "line=%%c"
setlocal enabledelayedexpansion
echo(!line:%ADMIN_HASH_LINE%:=! | findstr "^"
endlocal
)) > tmp_internal_users.yml

(for /f "delims=" %%d in ('type "tmp_internal_users.yml" ^| findstr /n "^"') do (
set "line=%%d"
setlocal enabledelayedexpansion
if !line:^%ADMIN_HASH_LINE%^=! neq !line! (
echo !line!
) else (
echo !line!
echo hash: "!HASHED_ADMIN_PASSWORD!"
)
endlocal
)) > "%INTERNAL_USERS_FILE%"

del tmp_internal_users.yml

:: network.host
>nul findstr /b /c:"network.host" "%OPENSEARCH_CONF_FILE%" && (
echo network.host already present
Expand Down
25 changes: 25 additions & 0 deletions tools/install_demo_configuration.sh
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ else
echo "DEBUG: basedir does not exist"
fi
OPENSEARCH_CONF_FILE="$BASE_DIR/config/opensearch.yml"
INTERNAL_USERS_FILE = "$BASE_DIR/config/internal_users.yml"
ADMIN_PASSWORD_FILE="$BASE_DIR/config/admin_password.txt"
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
OPENSEARCH_BIN_DIR="$BASE_DIR/bin"
OPENSEARCH_PLUGINS_DIR="$BASE_DIR/plugins"
OPENSEARCH_MODULES_DIR="$BASE_DIR/modules"
Expand Down Expand Up @@ -387,6 +389,29 @@ echo 'plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_
echo 'plugins.security.system_indices.enabled: true' | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null
echo 'plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*", ".opendistro-job-scheduler-lock"]' | $SUDO_CMD tee -a "$OPENSEARCH_CONF_FILE" > /dev/null

ADMIN_PASSWORD=$(cat "$ADMIN_PASSWORD_FILE")
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved

if [ -z "$ADMIN_PASSWORD" ]; then
if [ -n "$ENV_ADMIN_PASSWORD" ]; then
ADMIN_PASSWORD="$ENV_ADMIN_PASSWORD"
else
echo "Unable to find admin password for cluster, please run `export ENV_ADMIN_PASSWORD=<your_password>>` or create a file {OPENSEARCH_ROOT}/admin_password.txt with a single line that contains the password followed by a newline"
exit 1
fi
fi

salt=$(openssl rand -hex 8)

# Generate the hash using OpenBSD-style Blowfish-based bcrypt
HASHED_ADMIN_PASSWORD=$(openssl passwd -bcrypt -salt $salt "$ADMIN_PASSWORD")

# Clear the clearTextPassword variable
unset ADMIN_PASSWORD

ADMIN_HASH_LINE=$(grep -n 'admin:' "$INTERNAL_USERS_FILE" | cut -f1 -d:)

sed -i "${ADMIN_HASH_LINE}s/.*/ hash: \"$HASHED_ADMIN_PASSWORD\"/" "$INTERNAL_USERS_FILE"

#network.host
if $SUDO_CMD grep --quiet -i "^network.host" "$OPENSEARCH_CONF_FILE"; then
: #already present
Expand Down
Loading