Monitoring

Monitoring datavirtuality Server

You can see the server log in realtime as described in the section 'Logging'. Use tail (or Tail for Windows if you are not using a Linux system) to read the activity stream from the command line. Additionally, you can monitor the Java process behind the server. We will give a brief description about the tools usable if you want to monitor the Java process with respect to performance counters or network communication.

Process Monitoring on Linux

Linux offers some basic and useful tools out of the box. There are four tools which help monitor and trace the server process: top, htop, netstat and tail. The tools top and htop allow you to follow the resource consumption of the programs and services that are running on the machine. Maybe the htop tool is not yet installed (top most certainly is), but you can quickly get it via apt, yum or the package manager you use on the system. The netstat command line tool is useful to see the TCP and UDP connections of your system and you can combine it with grep to filter the output table. After you noted the process id (PID) of the datavirtuality Server process, you can execute 'netstat -aon | grep %PID%' and receive all bindings that the server is currently using for connections as well as the listening port and ip. The last tool, tail, opens a file and will automatically refresh and show you the last lines of the file you are watching. This is an excellent way of real-time auditing the server and checking for anomalies. All the tools are command line tools and can be started from a terminal session.

Overview of typical monitoring tools

• • top and htop
  • netstat  (use 'netstat -aon | grep %PID%' to get all network data from a specific process)
  • tail

Process Monitoring on Windows

On Windows you can use the standard tools that are provided by the system. Most commonly used, the Task-Manager can be used to check whch processes are running and what their resource consumption is. Go to the Resource Monitor to have a graphical visualization of different performance counters. Not only can the CPU utilization and RAM usage be audited, but the network rate and disc performance is also displayable. If you want to some information but do not care about visualization in form of small graphs, you can modify the Task-Manager slightly to give you more details about the running processes. This is also very useful when you want to have the process id (PID) that is being used for datavirtuality Server. For example, you can use this information to check which ports datavirtuality Server is listening on and whether there are any open connections to remote hosts with the command line tool netstat. Simply start the Task-Manager and go to the Processes tab. Then click on the menu item View and chose Select columns. Now you have a variety of supplemental columns to monitor you server's performance. After you have checked the PID of the datavirtuality Server process, you can use the command line tool netstat and use the parameters -aon to get a list of all processes with their TCP and UDP connections.

Overview of typical monitoring tools

• • Task-Manager and  Resource Monitor (Windows built-in for performance auditing)
  • netstat (Windows built-in command line tool to check and manage network connections and configurations)
  • Tail for Windows (Third party software to watch log files in real-time)

Local and Remote Monitoring using JConsole

If you want to monitor the Java Virtual Machine that encapsulates the datavirtuality Server in detail, use the tool JConsole. It is part of the Java JDK available on both Linux and Windows systems, once the JDK is installed. Typically, after the installation you have a 'bin' folder somewhere on you system (e.g. /usr/lib/jvm/java-7-oracle/bin on Linux or C:\Program Files\Java\jdk1.7.0_60\bin on windows) where the jconsole is located. Subsequent to obtaining the PID of datavirtuality Server via top (Linux) or Task-Manager (Windows), you can run the jconsole via one of the following commands:

It also possible to execute jconsole and enter the connection parameter by hand. Note that the PID-based connection is only possible for local instances. Any remote management requires the connection via host and port.

System Tables to Monitor

As it is possible to access the system tables and procedures via jdbc or odbc, the most monitoring systems can easily check them using simple SQL queries. The following tables could possibly a useful area to monitor datavirtuality Server.

SYSADMIN.ScheduleJobRun

• to check, if jobs failed, you can check for status = 'failed' and get the failureReason

• to look for long running queries, you can check for example queries which are running longer than 60 seconds with

   

  SELECT sqlCommand,startTime,endTime,status,failureReason FROM SYSADMIN.ScheduleJobRun WHERE TIMESTAMPDIFF( SQL_TSI_SECOND, startTime, endTime ) > 60

  See Date_Time_Funktions in the ReferenceManual for other time intervals to check.

See Jobs and Schedules in the ReferenceManual for more information.

SYSADMIN.Queries

Monitor all queries that have been issued against Data Virtuality Server. To check for long queries use, for example queries which are running longer than 60 seconds use:

See Date_Time_Funktions in the ReferenceManual for other time intervals to check.

To check be careful to monitor every failing query, because every wrong query, also mistyped queries by users will be in the Query list.

SYSADMIN.getAllDataSourcesMetadataDiff()

returns metadata difference for all data sources, to check if Source Systems changed the metadata.