What is the big deal, you may ask? Well, read on and all shall be revealed, intrepid reader ; )
As I mentioned an earlier post, our new test-runner – dbqp – allows us to define testing ‘modes’ which all utilize the same system and server management code.
One only has to define a testManager (what does a test look like / how to organize tests) and a testExecutor (how to execute / evaluate a test). The aim is for dbqp to be a one-stop shop for test execution and to provide a clean and simple way to manage and expand this.
I have just added –mode=randgen to the test-runner. The random query generator is a significant part of Drizzle’s testing strategy and we use a large number of tests with the tool. Currently, they are executed and managed by drizzle-automation / Jenkins in our build system, but there has not been an easy way for users to execute these tests (outside of installing / learning drizzle automation or the randgen)…until now >; )
Documentation including requirements and setup instructions for the randgen can be found here
For the cost of about 300 lines of code (the new manager/executor files), we can now execute our randgen tests much the same way as we do our drizzletest suite!
Each testcase is a simple .cnf file that defines a few crucial elements:
[test_info] comment = starts up a master-slave setup. Still needs some validation work [test_command] command = ./gentest.pl –gendata=conf/drizzle/drizzle.zz –grammar=conf/drizzle/optimizer_subquery_drizzle.yy –queries=10 –threads=1 [test_servers] servers = [[–innodb.replication-log=true],[–plugin-add=slave –slave.config-file=$MASTER_SERVER_SLAVE_CONFIG]]The test_command is the command line that is passed to the randgen tool itself. We have created most of these tests and have put together the command lines that serve our testing needs. By organizing our tests in this fashion, it is now easy for anyone to use the tool to perform meaningful tests.
The test_servers section is interesting – it is a simple list of python lists. Each sublist contains a set of server options that a server will need. Servers are started in-order with the first defined server being considered the ‘master_server’. As we can see in the example to test the new slave plugin, the files are amazingly simple and clean.
Some other added benefits are that dbqp’s –valgrind and –gdb options work the same with randgen tests as they would with drizzletest cases. Addtionally, we will be able to incorporate the code coverage provided by the randgen in our gcov reports.
One final little feature worth noting is the –gendata option.
The randgen provides a feature called the random data generator / gendata.pl
By passing it a configuration file that defines a set of test tables (see the forge documentation for further details), it will connect to a server, create the test tables, and populate them according to specifications.
While I have other plans for this, one cool use is for ad-hoc testing. By passing a simple command line + the use of –start-and-exit, one can have a nicely populated test database to play with:
./dbqp –mode=randgen –randgen-path=/home/user/repos/randgen –gendata=/home/user/repos/randgen/conf/drizzle/drizzle.zz –start-and-exit Setting –no-secure-file-priv=True for randgen mode… <snip> 25 Feb 2011 13:28:23 INFO: Using testing mode: randgen 25 Feb 2011 13:28:23 INFO: Processing test suites… 25 Feb 2011 13:28:23 INFO: Found 18 test(s) for execution 25 Feb 2011 13:28:23 INFO: Creating 1 testbot(s) 25 Feb 2011 13:28:23 INFO: Taking clean db snapshot… 25 Feb 2011 13:28:23 INFO: testbot0 server: 25 Feb 2011 13:28:23 INFO: NAME: server0 25 Feb 2011 13:28:23 INFO: MASTER_PORT: 9306 25 Feb 2011 13:28:23 INFO: DRIZZLE_TCP_PORT: 9307 25 Feb 2011 13:28:23 INFO: MC_PORT: 9308 25 Feb 2011 13:28:23 INFO: PBMS_PORT: 9309 25 Feb 2011 13:28:23 INFO: RABBITMQ_NODE_PORT: 9310 25 Feb 2011 13:28:23 INFO: VARDIR: drizzle/tests/workdir/testbot0/server0/var 25 Feb 2011 13:28:23 INFO: STATUS: 1 # 2011-02-25T13:28:23 Default schema: test # 2011-02-25T13:28:23 Executor initialized, id GenTest::Executor::Drizzle 2011.02.2198 () # 2011-02-25T13:28:23 # Creating Drizzle table: test.A; engine: ; rows: 0 . # 2011-02-25T13:28:23 # Creating Drizzle table: test.B; engine: ; rows: 0 . # 2011-02-25T13:28:23 # Creating Drizzle table: test.C; engine: ; rows: 1 . # 2011-02-25T13:28:23 # Creating Drizzle table: test.D; engine: ; rows: 1 . # 2011-02-25T13:28:23 # Creating Drizzle table: test.AA; engine: ; rows: 10 . # 2011-02-25T13:28:23 # Creating Drizzle table: test.BB; engine: ; rows: 10 . # 2011-02-25T13:28:24 # Creating Drizzle table: test.CC; engine: ; rows: 100 . # 2011-02-25T13:28:24 # Creating Drizzle table: test.DD; engine: ; rows: 100 . 25 Feb 2011 13:28:24 INFO: User specified –start-and-exit. dbqp.py exiting and leaving servers running… ../client/drizzle -uroot -p9306 test Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the Drizzle client.. Commands end with ; or \g. Your Drizzle connection id is 3 Connection protocol: mysql Server version: 2011.02.2198 Source distribution (dbqp_randgen_updates) Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the buffer. drizzle> show tables; +—————-+ | Tables_in_test | +—————-+ | A | | AA | | B | | BB | | C | | CC | | D | | DD | +—————-+ 8 rows in set (0.001614 sec) drizzle> select count(*) from dd; +———-+ | count(*) | +———-+ | 100 | +———-+ 1 row in set (0.000614 sec)Please give it a try if you feel so inclined. We will be working to integrate this better into our build and test systems (Jenkins / make targets / etc). And remember ./dbqp –mode=cleanup for quick and easy cleanup when you are done playing! ; )