Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Key cloak User Management #378

Open
wants to merge 45 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
781c973
add token, register, ban, unban
eelcovdw Oct 8, 2024
6a9addd
merge dev
eelcovdw Oct 9, 2024
1756718
add email
eelcovdw Oct 9, 2024
aa8c8c0
merge settings
eelcovdw Oct 9, 2024
7f8f2de
update endpoint, use settings
eelcovdw Oct 9, 2024
9fc970d
add jwt auth
eelcovdw Oct 9, 2024
a9b5ccf
save progress
teo-milea Nov 8, 2024
b44fabb
syftbox/
teo-milea Nov 12, 2024
11c0272
Merge branch 'main' of github.com:OpenMined/syft into eelco/user-mana…
teo-milea Nov 13, 2024
20286e7
moved some functions after merge
teo-milea Nov 13, 2024
52e2e5b
remove redundant variables
abyesilyurt Nov 13, 2024
cfd1a41
add authentication to sync endpoints
abyesilyurt Nov 13, 2024
cfb0f56
fix api tests
abyesilyurt Nov 13, 2024
81c2b9e
fix unit tests
abyesilyurt Nov 13, 2024
368015c
client sends email on every request
abyesilyurt Nov 13, 2024
ff2de73
test can run against the keycloak server
abyesilyurt Nov 13, 2024
020df53
add email verification checks
abyesilyurt Nov 13, 2024
f8fe205
refactor email verification check
abyesilyurt Nov 13, 2024
81efcc2
clean up, new ux
teo-milea Nov 13, 2024
c4b65a2
added just args and remove some old code
teo-milea Nov 13, 2024
f173915
justfile run client args
teo-milea Nov 13, 2024
b422210
save uv lock
teo-milea Nov 14, 2024
a9c344a
Merge branch 'main' of github.com:OpenMined/syft into eelco/user-mana…
teo-milea Nov 14, 2024
c8b7e12
Merge branches 'aziz/keycloak' and 'eelco/user-management' of github.…
abyesilyurt Nov 14, 2024
153ef29
fix token
teo-milea Nov 14, 2024
1bd17b9
Merge branch 'eelco/user-management' of github.com:OpenMined/syft int…
abyesilyurt Nov 14, 2024
2f2e1a2
client auth working
abyesilyurt Nov 14, 2024
b45b0f9
Merge pull request #381 from OpenMined/aziz/keycloak
abyesilyurt Nov 14, 2024
3ee03e7
remove unintentional changes
abyesilyurt Nov 14, 2024
cbf4678
fix duplicate pre-commit entry
abyesilyurt Nov 14, 2024
bb79b40
remove submodules
abyesilyurt Nov 14, 2024
25287fc
remove unused code
abyesilyurt Nov 14, 2024
d687921
remove unused code
abyesilyurt Nov 14, 2024
f5f0a1b
working register
teo-milea Nov 15, 2024
8e476d4
Merge branch 'eelco/user-management' of github.com:OpenMined/syft int…
teo-milea Nov 15, 2024
2e8e80b
fix path
teo-milea Nov 15, 2024
2a6e0c5
add password
abyesilyurt Nov 15, 2024
3d7a3b2
added scope=openied
teo-milea Nov 18, 2024
622fb3d
add token validation to client
abyesilyurt Nov 18, 2024
8a90d73
moved auth check to somewhere else
abyesilyurt Nov 18, 2024
e37e445
fix email auth
abyesilyurt Nov 18, 2024
ff9fcc5
finished ux/removed password from config file
teo-milea Nov 18, 2024
ab92a8f
Merge branch 'eelco/user-management' of github.com:OpenMined/syft int…
teo-milea Nov 18, 2024
8d1091c
fix
abyesilyurt Nov 18, 2024
bcc372b
fix register
abyesilyurt Nov 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ run-server port="5001" uvicorn_args="":

# Run a local syftbox client on any available port between 8080-9000
[group('client')]
run-client name port="auto" server="http://localhost:5001":
run-client name port="auto" server="http://localhost:5001" register="false" reset_password="false":
#!/bin/bash
set -eou pipefail

Expand All @@ -59,12 +59,19 @@ run-client name port="auto" server="http://localhost:5001":
DATA_DIR=.clients/$EMAIL
mkdir -p $DATA_DIR

echo -e "Email : {{ _green }}$EMAIL{{ _nc }}"
echo -e "Client : {{ _cyan }}http://localhost:$PORT{{ _nc }}"
echo -e "Server : {{ _cyan }}{{ server }}{{ _nc }}"
echo -e "Data Dir : $DATA_DIR"
echo -e "Email : {{ _green }}$EMAIL{{ _nc }}"
echo -e "Client : {{ _cyan }}http://localhost:$PORT{{ _nc }}"
echo -e "Server : {{ _cyan }}{{ server }}{{ _nc }}"
echo -e "Data Dir : $DATA_DIR"
echo -e "Register : {{ register }}"
echo -e "Reset Password : {{ reset_password }}"

OTHER_OPTS=""
if [[ "{{ register }}" == "true" ]]; then OTHER_OPTS="--register "; fi
if [[ "{{ reset_password }}" == "true" ]]; then OTHER_OPTS="$OTHER_OPTS --reset_password "; fi


uv run syftbox/client/cli.py --config=$DATA_DIR/config.json --data-dir=$DATA_DIR --email=$EMAIL --port=$PORT --server={{ server }} --no-open-dir
uv run syftbox/client/cli.py --config=$DATA_DIR/config.json --data-dir=$DATA_DIR --email=$EMAIL --port=$PORT --server={{ server }} --no-open-dir $OTHER_OPTS

# ---------------------------------------------------------------------------------------------------------------------

Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ version = "0.2.4"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.9"

# add using uv add <pip package>
dependencies = [
"fastapi>=0.114.0",
"uvicorn>=0.30.6",
Expand Down
14 changes: 13 additions & 1 deletion syftbox/client/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,16 @@
is_flag=True,
help="Enable verbose mode",
)
REGISTER_OPTS = Option(
"--register",
is_flag=True,
help="Register new user",
)
RESET_PASS_OPTS = Option(
"--reset_password",
is_flag=True,
help="Register new user",
)

# report command opts
REPORT_PATH_OPTS = Option(
Expand All @@ -79,6 +89,8 @@ def client(
port: Annotated[int, PORT_OPTS] = DEFAULT_PORT,
open_dir: Annotated[bool, OPEN_OPTS] = True,
verbose: Annotated[bool, VERBOSE_OPTS] = False,
register: Annotated[bool, REGISTER_OPTS] = False,
reset_password: Annotated[bool, RESET_PASS_OPTS] = False
):
"""Run the SyftBox client"""

Expand All @@ -99,7 +111,7 @@ def client(
rprint(f"[bold red]Error:[/bold red] Client cannot start because port {port} is already in use!")
raise Exit(1)

client_config = setup_config_interactive(config_path, email, data_dir, server, port)
client_config = setup_config_interactive(config_path, email, data_dir, server, port, register, reset_password)
log_level = "DEBUG" if verbose else "INFO"
code = run_client(client_config=client_config, open_dir=open_dir, log_level=log_level)
raise Exit(code)
Expand Down
48 changes: 47 additions & 1 deletion syftbox/client/cli_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,28 @@

from pathlib import Path

from loguru import logger
from rich import print as rprint
from rich.prompt import Confirm, Prompt

from syftbox.lib.client_config import SyftClientConfig
from syftbox.lib.constants import DEFAULT_DATA_DIR
from syftbox.lib.exceptions import ClientConfigException
from syftbox.lib.keycloak import get_token
from syftbox.lib.validators import DIR_NOT_EMPTY, is_valid_dir, is_valid_email

__all__ = ["setup_config_interactive"]


def setup_config_interactive(config_path: Path, email: str, data_dir: Path, server: str, port: int) -> SyftClientConfig:
def setup_config_interactive(
config_path: Path,
email: str,
data_dir: Path,
server: str,
port: int,
register: bool,
reset_password: bool,
) -> SyftClientConfig:
"""Setup the client configuration interactively. Called from CLI"""

config_path = config_path.expanduser().resolve()
Expand All @@ -37,20 +47,42 @@ def setup_config_interactive(config_path: Path, email: str, data_dir: Path, serv
if not email:
email = prompt_email()

password = register_password() if register else login_password()

access_token = get_token(email, password)

# create a new config with the input params
conf = SyftClientConfig(
path=config_path,
sync_folder=data_dir,
email=email,
server_url=server,
port=port,
access_token=access_token,
)
else:
if conf.access_token is None:
logger.info("No access token found in the config. Please login again.")
pwd = login_password()
conf.access_token = get_token(conf.email, pwd)
if server and server != conf.server_url:
conf.set_server_url(server)
if port != conf.client_url.port:
conf.set_port(port)

if reset_password:
if register:
rprint("You cannot register and reset password at the same time!")
exit()
else:
new_password = register_password()
resp = reset_password(conf.user_id, new_password, conf.access_token)
if resp.status_code == 204:
rprint("[bold]Password reset succesful![/bold]")
else:
rprint("[bold red]An error occured![/bold red] '{resp.text}'")
exit()

# DO NOT SAVE THE CONFIG HERE.
# We don't know if the client will accept the config yet
return conf
Expand Down Expand Up @@ -85,3 +117,17 @@ def prompt_email() -> str:
rprint(f"[bold red]Invalid email[/bold red]: '{email}'")
continue
return email


def register_password() -> str:
while True:
password = Prompt.ask("[bold]Enter your password[/bold]")
verify_password = Prompt.ask("[bold]Verify your password[/bold]")
if password == verify_password:
break
rprint("[bold red]Passwords don't match! Please try again.[/bold red]")
return password


def login_password() -> str:
return Prompt.ask("[bold]Password:[/bold]")
40 changes: 34 additions & 6 deletions syftbox/client/client2.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
from syftbox.lib.datasite import create_datasite
from syftbox.lib.exceptions import SyftBoxException
from syftbox.lib.ignore import IGNORE_FILENAME
from syftbox.lib.keycloak import get_token
from syftbox.lib.workspace import SyftWorkspace

SCRIPT_DIR = Path(__file__).parent
Expand Down Expand Up @@ -54,7 +55,17 @@ def __init__(self, config: SyftClientConfig, log_level: str = "INFO", **kwargs):

self.workspace = SyftWorkspace(self.config.data_dir)
self.pid = PidFile(pidname="syftbox.pid", piddir=self.workspace.data_dir)
self.server_client = httpx.Client(base_url=str(self.config.server_url), follow_redirects=True)

self.server_client = httpx.Client(
base_url=str(self.config.server_url),
follow_redirects=True,
# We are sending email along with the bearer token
# to support fallback to email based authentication in case of keycloak failure
# or local development without keycloak
# To configure the server to use bypass keycloak authentication
# set the environment variable SYFTBOX_NO_AUTH=1
headers={"email": self.config.email, "Authorization": f"Bearer {self.config.access_token}"},
)

# kwargs for making customization/unit testing easier
# this will be replaced with a sophisticated plugin system
Expand Down Expand Up @@ -85,7 +96,7 @@ def app_runner(self):
@property
def is_registered(self) -> bool:
"""Check if the current user is registered with the server"""
return bool(self.config.token)
return bool(self.config.access_token)

@property
def datasite(self) -> Path:
Expand Down Expand Up @@ -156,11 +167,11 @@ def register_self(self):
if self.is_registered:
return
try:
token = self.__register_email()
access_token = self.__register_email()
# TODO + FIXME - once we have JWT, we should not store token in config!
# ideally in OS keychain (using keyring) or
# in a separate location under self.workspace.plugins
self.config.token = str(token)
self.config.access_token = access_token
self.config.save()
logger.info("Email registration successful")
except Exception as e:
Expand All @@ -186,9 +197,16 @@ def __run_local_server(self):

def __register_email(self) -> str:
# TODO - this should probably be wrapped in a SyftCacheServer API?
response = self.server_client.post("/register", json={"email": self.config.email})
payload = {
"email": self.config.email,
"password": self.config.password,
"firstName": "",
"lastName": "",
}
response = self.server_client.post("/users/register", json=payload)
response.raise_for_status()
return response.json().get("token")
token = get_token(self.config.email, self.config.password)
return token

def __enter__(self):
return self
Expand Down Expand Up @@ -333,6 +351,16 @@ def run_client(

try:
client = SyftClient(client_config, log_level=log_level)

# authentication check
response = client.server_client.post("/sync/datasites")
try:
response.raise_for_status()
except httpx.HTTPStatusError as e:
logger.error(f"Failed to authenticate with the server: {e}")
if response.status_code == 401:
logger.info("Please login to refresh your session.")

# we don't want to run migration if another instance of client is already running
bool(client.check_pidfile()) and run_migration(client_config)
(not syftbox_env.DISABLE_ICONS) and client.copy_icons()
Expand Down
5 changes: 5 additions & 0 deletions syftbox/client/env.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ class SyftEnvVars(BaseSettings):
CLIENT_CONFIG_PATH: Path = Field(default=DEFAULT_CONFIG_PATH)
"""Path to the client configuration file."""

ACCESS_TOKEN: str = Field(default="")
"""Access token for the datasite."""

KEYCLOAK_ADMIN_TOKEN: str = Field(default="")

model_config = SettingsConfigDict(env_file=".env", env_prefix="SYFTBOX_")


Expand Down
16 changes: 14 additions & 2 deletions syftbox/lib/client_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

from syftbox.lib.constants import DEFAULT_CONFIG_PATH, DEFAULT_DATA_DIR, DEFAULT_SERVER_URL
from syftbox.lib.exceptions import ClientConfigException
from syftbox.lib.keycloak import get_user_from_token
from syftbox.lib.types import PathLike, to_path

__all__ = ["SyftClientConfig"]
Expand Down Expand Up @@ -49,8 +50,13 @@ class SyftClientConfig(BaseModel):
email: EmailStr = Field(description="Email address of the user")
"""Email address of the user"""

token: Optional[str] = Field(default=None, description="API token for the user")
"""API token for the user"""
token: Optional[str] = Field(
default=None, description="Depracated: Use access_token instead. API token for the user", deprecated=True
)
"""Depracated: Use access_token instead. API token for the user"""

access_token: str = Field(default=None, description="Access token for the user")
"""Access token for the user"""

# WARN: we don't need `path` to be serialized, hence exclude=True
path: Path = Field(exclude=True, description="Path to the config file")
Expand All @@ -62,6 +68,12 @@ def port_to_url(cls, val):
return f"http://127.0.0.1:{val}"
return val

@property
def user_id(self):
if self.access_token:
return get_user_from_token(self.access_token)["sub"]
return None

@field_validator("token", mode="before")
def token_to_str(cls, v):
if not v:
Expand Down
Loading
Loading