cockroachdb / cockroachdb/movr
Error in running workload via roachprod steps
- Dominant language
- Python
- Stars
- 37
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
In running through the [partitioning demo](https://cockroachlabs.atlassian.net/wiki/spaces/ENG/pages/109314241/Geo-Partitioning+a+Multi-Region+Cluster) I'm hitting errors after this step:
`sudo docker run -it --rm cockroachdb/movr:19.03.2 --app-name "movr-west" --url "postgres://root@10.138.0.22:26257/movr?sslmode=disable" --num-threads=15 run --city="seattle"`
Here is the errors:
```
Unable to find image 'cockroachdb/movr:19.03.2' locally
19.03.2: Pulling from cockroachdb/movr
54f7e8ac135a: Pull complete
d6341e30912f: Pull complete
087a57faf949: Pull complete
5d71636fb824: Pull complete
0c1db9598990: Pull complete
bfb904e99f24: Pull complete
78a3d3a96a32: Pull complete
885a0ed92c89: Pull complete
dd7cc9ace242: Pull complete
73f819bc74c5: Pull complete
e5b919159286: Pull complete
35399a9c50c3: Pull complete
65c4d0c6af6d: Pull complete
506364b121bd: Pull complete
388fa79ed397: Pull complete
af50ea530dd3: Pull complete
Digest: sha256:615e43f2334fea6c2d1342f9ede82a23164813645dfbfbd700734a6b867a0732
Status: Downloaded newer image for cockroachdb/movr:19.03.2
[INFO] (MainThread) connected to movr database @ postgres://root@10.138.0.22:26257/movr?sslmode=disable
[INFO] (MainThread) simulating movr load for cities ['seattle']
[INFO] (MainThread) warming up....
[INFO] (MainThread) starting load
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: relation "rides" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "./loadmovr.py", line 110, in simulate_movr_load
movr.end_ride(ride['city'], ride['id'])
File "/usr/src/app/movr.py", line 60, in end_ride
run_transaction(sessionmaker(bind=self.engine), lambda session: end_ride_helper(session, city, ride_id))
File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 31, in run_transaction
return _txn_retry_loop(session, callback)
File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 75, in _txn_retry_loop
ret = callback(conn)
File "/usr/src/app/movr.py", line 60, in
run_transaction(sessionmaker(bind=self.engine), lambda session: end_ride_helper(session, city, ride_id))
File "/usr/src/app/movr.py", line 53, in end_ride_helper
ride = session.query(Ride).filter_by(city=city, id=ride_id).first()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2979, in first
ret = list(self[0:1])
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2771, in __getitem__
return list(res)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3081, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3106, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 276, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "rides" does not exist
[SQL: 'SELECT rides.id AS rides_id, rides.city AS rides_city, rides.vehicle_city AS rides_vehicle_city, rides.rider_id AS rides_rider_id, rides.vehicle_id AS rides_vehicle_id, rides.start_address AS rides_start_address, rides.end_address AS rides_end_address, rides.start_time AS rides_start_time, rides.end_time AS rides_end_time, rides.revenue AS rides_revenue \nFROM rides \nWHERE rides.city = %(city_1)s AND rides.id = %(id_1)s \n LIMIT %(param_1)s'] [parameters: {'city_1': 'seattle', 'id_1': 'a75e3d50-7292-4a4b-a073-77625ca40c3f', 'param_1': 1}] (Background on this error at: http://sqlalche.me/e/f405)
Exception in thread Thread-11:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: relation "rides" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "./loadmovr.py", line 110, in simulate_movr_load
movr.end_ride(ride['city'], ride['id'])
File "/usr/src/app/movr.py", line 60, in end_ride
run_transaction(sessionmaker(bind=self.engine), lambda session: end_ride_helper(session, city, ride_id))
File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 31, in run_transaction
return _txn_retry_loop(session, callback)
File "/usr/local/lib/python3.7/site-packages/cockroachdb/sqlalchemy/transaction.py", line 75, in _txn_retry_loop
ret = callback(conn)
File "/usr/src/app/movr.py", line 60, in
run_transaction(sessionmaker(bind=self.engine), lambda session: end_ride_helper(session, city, ride_id))
File "/usr/src/app/movr.py", line 53, in end_ride_helper
ride = session.query(Ride).filter_by(city=city, id=ride_id).first()
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2979, in first
ret = list(self[0:1])
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 2771, in __getitem__
return list(res)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3081, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3106, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1458, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 276, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "rides" does not exist
[SQL: 'SELECT rides.id AS rides_id, rides.city AS rides_city, rides.vehicle_city AS rides_vehicle_city, rides.rider_id AS rides_rider_id, rides.vehicle_id AS rides_vehicle_id, rides.start_address AS rides_start_address, rides.end_address AS rides_end_address, rides.start_time AS rides_start_time, rides.end_time AS rides_end_time, rides.revenue AS rides_revenue \nFROM rides \nWHERE rides.city = %(city_1)s AND rides.id = %(id_1)s \n LIMIT %(param_1)s'] [parameters: {'city_1': 'seattle', 'id_1': 'c5ada0f1-092d-436b-8e38-dbd1a57afac6', 'param_1': 1}] (Background on this error at: http://sqlalche.me/e/f405)
Exception in thread Thread-7:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: relation "rides" does not exist
```
I'm filing this for now and will look into it more tomorrrow.
cc @jseldess and @ericharmeling
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the partitioning demo command and the referenced loadmovr.py and movr.py locations, then reproduce the workload against the stated database URL. Trace where the rides relation is expected to be created or accessed; done means the workload runs without the reported missing-relation error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, postgres, python
- Domain
- databases, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100