Repository navigation
[Bug]: Python streaming LoadTests failing since July 8 #22422
Description
What happened?
https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/
06:01:37 [--input_options INPUT_OPTIONS] [--timeout_ms TIMEOUT_MS]
06:01:37 [--iterations ITERATIONS]
06:01:37 [--number_of_counter_operations NUMBER_OF_COUNTER_OPERATIONS]
06:01:37 [--number_of_counters=0 NUMBER_OF_COUNTERS=0]
06:01:37 pardo_test.py: error: argument --number_of_counters=0: expected one argument
Issue Priority
Priority: 1
Issue Component
Component: build-system
Activity
TheNeuralBit commentedon Jul 23, 2022
Looks like this started with 20949a9,
TheNeuralBit commentedon Jul 23, 2022
This is a gross one.
The problem is the space in the argument
--experiments=use_runner_v2, shuffle_mode=applianceshuffle_mode=appliance is being interpreted as a separate argument because there's a space and it's not quoted. Then after we parse the known arguments we're left with the unknown_args:
The logic interprets shuffle_mode=appliance as the argument for an option with name
number_of_counters=0.We can fix by changing
use_runner_v2, shuffle_mode=appliance,use_runner_v2,shuffle_mode=appliance, or nmore generally by adding quoting logic in LoadTestsBuilder.groovy. We should also adjust the retain_unknown_options logic. It's bad that it assumes the arguments are consecutive.tvalentyn commentedon Jul 23, 2022
cc: @robertwb