Mastering Tungsten Replicator Series: Command Line Superpowers

The Tungsten Replicator is an extraordinarily powerful and flexible tool capable of moving vast volumes of data from source to target. The Tungsten Replicator is embedded into the Tungsten Clustering solution to provide core replication services, replacing native MySQL replication.

In this blog post we will discuss a few of the commands that will make your life easier.

There are times when it is important to understand the inner state of the Replicator, for example when troubleshooting and for performance tuning.

Tungsten Replicator provides multiple command-line tools to inspect the inner workings of the replication pipelines, and the key commands are trepctl and thl.

The trepctl command has many available functions, and is used for:

• Putting the replicator online or offline:
  trepctl online and trepctl offline
• Performing backup and restore operations:
  trepctl backup and trepctl restore
• Skipping events in the THL in the event of an issue:
  trepctl online -skip-seqno {comma-separated list of sequence numbers}
• Getting status and active configuration information:
  trepctl {services|status|perf|qs}

As of software version 5.2.0, the trepctl command has been enhanced with two new features to provide both quick (qs) and deep (perf) status information.

Let's start with the Quick Status feature. The trepctl qs command provides a quicker, simpler, status display for the replicator showing only the critical information in a human-readable form. For example:

shell> trepctl qs
State: west Online for 4387826.891s, running for 5446160.734s
Sequence: 137530235 last applied, 1 transactions behind (135658655-137530236 stored) estimate 0.03s before synchronization

For more details, please visit the docs page at http://docs.continuent.com/tungsten-replicator-6.0/cmdline-tools-trepctl-command-qs.html

Next up is the Performance feature. The trepctl perf outputs performance information on a stage by stage basis from the current replicator. The information has been reformatted and extracted from the existing replicator status, task and stage information available through other commands and requests, but reformatted and with values calculated to make identifying specific performance metrics quicker.

For example, on a typical extraction replicator:

shell> trepctl perf -r 2
Statistics since last put online 9265.385s ago
Stage       | Seqno | Latency |      Events |  Extraction |   Filtering |    Applying |       Other |       Total 
binlog-to-q |  1604 |  8.779s |          14 |     60.173s |      0.109s |      0.015s |      0.004s |     60.301s 
                         Avg time per Event |      4.298s |      0.008s |      0.000s |      0.001s |      4.307s
                           Filters in stage | colnames -> pkey
q-to-thl    |  1604 | 10.613s |          14 |     56.858s |      0.020s |      5.247s |      0.028s |     62.153s 
                         Avg time per Event |      4.061s |      0.001s |      0.002s |      0.375s |      4.440s
                           Filters in stage | enumtostring -> settostring

On an applier:

shell> trepctl perf -r 2
Statistics since last put online 38.418s ago
Stage         | Seqno | Latency |      Events |  Extraction |   Filtering |    Applying |       Other |       Total 
remote-to-thl |  3246 |  1.143s |          42 |     37.831s |      0.001s |      0.403s |      0.011s |     38.246s 
                           Avg time per Event |      0.901s |      0.000s |      0.000s |      0.010s |      0.911s
thl-to-q      |  3246 |  1.209s |        1654 |     37.113s |      0.005s |      1.090s |      0.098s |     38.306s 
                           Avg time per Event |      0.022s |      0.000s |      0.000s |      0.001s |      0.023s
q-to-dbms     |  3235 |  3.746s |        1644 |     22.226s |      0.019s |     15.242s |      0.338s |     37.825s 
                           Avg time per Event |      0.014s |      0.000s |      0.000s |      0.009s |      0.023s
                             Filters in stage | mysqlsessions -> pkey

For convenience, the performance display can be set to auto-refresh at a specific interval using the trepctl perf -r {seconds} command. This is especially useful during troubleshooting to identify where bottlenecks may be and where within the pipeline time is actually spent (i.e. copying over the network, applying to the target or filtering activities).

For more details, please visit the docs page at http://docs.continuent.com/tungsten-replicator-6.0/cmdline-tools-trepctl-command-perf.html

We will continue to cover topics of interest in our next "Mastering Tungsten Replicator Series" post...stay tuned!

Click here for more online information about Continuent solutions...

Want to learn more or run a POC? Contact us.