Update pants-plugins/uses_services to support checking for redis

[cognifloyd]

Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

This PR builds on #5864 and #5884 to further improve the DX (developer experience) by failing as early as possible if the development (or CI) environment does not have the required services.

This PR adds checks for redis, like #5864 added mongo checks and #5884 added rabbitmq checks.

Please see #5864 for a more detailed description of pants-plugins/uses_services.

is_redis_running.py script and the rule that runs it

is_redis_running.py is the part that checks to see if redis is running.

The pants plugin cannot import anything from our other st2* code, so is_redis_running.py is a minimal self-contained script that mirrors how st2 connects to redis. It should be as minimal as possible so that keeping it up-to-date with the core st2 code is not onerous.

The is_redis_running.py rule gets opened and run in a pex with the same rule logic as is_mongo_running.py and inspect_platform.py (see #5864).

Redis, like rabbitmq, embeds all auth stuff in the connection url. So, there are not as many settings to worry about as there were with mongo. For now, this only supports the url hard-coded in the tests. Here is where the rule defines the default url:

st2/pants-plugins/uses_services/redis_rules.py

Lines 47 to 62 in 06aedfa

	@dataclass(frozen=True)
	class UsesRedisRequest:
	"""One or more targets need a running redis service using these settings.
	The coord_* attributes represent the coordination settings from st2.conf.
	In st2 code, they come from:
	oslo_config.cfg.CONF.coordination.url
	"""
	# These config opts for integration tests are in:
	# conf/st2.dev.conf (copied to conf/st2.ci.conf)
	# TODO: for int tests: set url by either modifying st2.{dev,ci}.conf on the fly or via env vars.
	# with our version of oslo.config (newer are slower) we can't directly override opts w/ environment variables.
	coord_url: str = "redis://127.0.0.1:6379"

Here is the definition of the rule that runs is_redis_running.py:

st2/pants-plugins/uses_services/redis_rules.py

Lines 96 to 98 in 06aedfa

	async def redis_is_running(
	request: UsesRedisRequest, platform: Platform
	) -> RedisIsRunning:

So, when another rule Gets RedisIsRunning with a UsesRedisRequest, pants will also run the rule that generates Platform (described in #5864), and then it will run this is_redis_running rule.

The is_redis_running rule either returns RedisIsRunning() if it is running, or raises ServiceMissingError if it is not. By raising an error, this actually breaks a convention for pants rules. Exceptions stop everything and get shown to the user right away, and for most goals, pants wants to see some dataclass returned that has something like a succeeded boolean instead. But, we want to error as early as possible, so this breaks that convention on purpose.

wiring up the test goal so it runs is_redis_running when pytest runs on a target with the uses field.

The last piece that ties this all together is a rule that makes sure the is_redis_running rule runs before pytest runs (if pants is running it on a target with the uses field). Here is the definition of the redis_is_running_for_pytest rule:

st2/pants-plugins/uses_services/redis_rules.py

Lines 83 to 85 in 06aedfa

	async def redis_is_running_for_pytest(
	request: PytestUsesRedisRequest,
	) -> PytestPluginSetup:

This rule is very simple. It just triggers running the is_redis_running rule:

st2/pants-plugins/uses_services/redis_rules.py

Lines 86 to 89 in 06aedfa

	# this will raise an error if redis is not running
	_ = await Get(RedisIsRunning, UsesRedisRequest())
	return PytestPluginSetup()

This rule needs the PytestUsesRedisRequest which selects targets that have the uses field.

st2/pants-plugins/uses_services/redis_rules.py

Lines 70 to 76 in 06aedfa

	class PytestUsesRedisRequest(PytestPluginSetupRequest):
	@classmethod
	def is_applicable(cls, target: Target) -> bool:
	if not target.has_field(UsesServicesField):
	return False
	uses = target.get(UsesServicesField).value
	return uses is not None and "redis" in uses

This request will be made by the pants-internal pytest rules thanks to this UnionRule, which is a way to declare that our class "implements" the PytestPluginSetupRequest:

st2/pants-plugins/uses_services/redis_rules.py

Line 168 in 06aedfa

UnionRule(PytestPluginSetupRequest, PytestUsesRedisRequest),

If we need to add uses support to other targets, we will need to find similar UnionRules where we can inject our rule logic. For now, I've only wired this up so our uses rules will run for pytest.

[rush-skills]

LGTM. Though is redis only required for orquestra tests and runner builds?

[cognifloyd]

I don't know. They are needed at least for the orquesta tests. When we find other tests that need any of the services (mongo, rabbitmq, redis), then we can add uses= to the BUILD metadata for those tests too.