Tuesday, November 29, 2011

Table partitioning: Other databases (informix, oracle, teradata and Sql Server)

Table partitioning is neat feature introduced in DB2 V9, made life much easier for DBA's. Thought let's compare it with other databases. Here is what my findings, by no mean exhaustive or complete......



Informix:

Informix has both partitioned and non-partitioned index. But there is *no* key word in the CREATE INDEX to say whether it should be partitioned or non-partitioned.
The decision is based on the context. The CREATE INDEX can be successfully creating a partitioned or non-partitioned index or returning a failure.
Whether index is partitioned or not is independent of that of the base table.
Informix index has the 'attached' or 'detached' concepts, which has nothing to do with the table partition attach/detatch operation.
They mean Informix index pages and table data page are in the same object if an index is 'attached'.
When the index object is separate from that of the table, it's considered as 'detatched'.

The attached/detached is independent of whether the base table is partitioned or not.
During table partition attach, existing indexes on the to-be attached table will be dropped unless they match the index definitions
in the target table ( the partitioned table ). Any missing indexes will be automatically created for the newly attached partition during ATTACH.
Everything is done under the cover. ATTACH will not return until the build of all matching indexes on the new partition complete.

Informix doesn't support index reorg syntax.
Index is maintained by a dedicated engine thread , which monitor the index page usage behind the scene and maintenance based on some criteria.
Partition level ( both index and table ) stats are collected and used.

Oracle: 

Backup and recovery operations can be performed on individual partitions.

Oracle supports two types of partitioned index:
Local -- All index entries in a single partition will correspond to a single table partition (equi-partitioned).
They are created with the LOCAL keyword and support partition independence. Equi-partitioning allows oracle to be more efficient whilst devising query plans.
Global - Index in a single partition may correspond to multiple table partitions.
They are created with the GLOBAL keyword and do not support partition independence.
Global indexes can only be range partitioned and may be partitioned in such a fashion that they look equi-partitioned,
but Oracle will not take advantage of this structure. Global partioned indexes must be prefixed with the partitioning key.

The user can specify which tablespace each partition of the index is stored in.


SQL Server:

Supports local indexes which can be partitioned different from the table data. Partitioning works as the following:
Create a partition function which maps the rows of a table or index into partitions based on the values of a specified column.
Create a partition scheme which maps the partitions of a partitioned table or index to filegroups.
Create a table using a partition scheme defined (note a default filegroup can be specified, similar to the IN/INDEX IN/etc. clauses of create table for DB2 LUW).
Create an index using a partition scheme defined (does not have to match the table's partitioning scheme).
Uniqueness for UNIQUE indexes is only verified within a single partition.
The default if no partition scheme is specified for create index on a partitioned table is to create partitioned indexes partitioned the same as the data.

All indexes require a partitioning scheme on a partitioned table, but that doesn't exclude global indexes as we know them.
A global index can be created by using a partitioning function which encompasses the entire range for a column and
using a partitioning scheme which puts it in one filegroup.
ALTER PARTITION
Can only split or merge partitions as the means to add/drop partitions. This is an offline operation.
ALTER INDEX
Can be used to rebuild or reorganize indexes offline or online for any of: one index, all indexes on the table, one index partition.

Teradata
Does not support local indexes


Wednesday, October 19, 2011

Combining DB2 and OS Diagnostics Part 3

Combining DB2 and OS Diagnostics Part 2

A simple methodology on Unix systems to approach a problem scenario:


On UNIX, problem investigation often starts with the following:
· Check vmstat or iostat output for the amount of system and user CPU being consumed
versus wait and idle time. Wait time usually, but not always, means waiting on I/O. If
system CPU is equal to or greater than user CPU, this almost always indicates a problem.
· Check the top CPU-using processes by ordering ps output by descending CPU penalty (C)
and memory usage (RSS and SZ columns). For example:
ps -elf | sort +5 -rn
· Are the same processes near the top? Is the CPU penalty staying in the 60-120 range? This
would mean these processes are constantly consuming CPU cycles and need to be
investigated.

On Windows, check for the top CPU and memory users by ordering the output on those columns.
I/O is more difficult to judge without getting perfmon output.

Where my DB2 database spending time


In order to identify where DB2 is spending most of it's time.

In 9.7 new in memory monitoring functions has been introduced.
One of the most interesting and useful feature I found, now you can easily determine where DB2 is spending most of its time.

You can use this SQL, V9.7 onward.

select u.metric_name,
u.parent_metric_name,
sum(u.total_time_value),
sum(u.count)
from table(mon_get_service_subclass_details(null,null,-2)) as t,
table(mon_format_xml_times_by_row(t.details)) as u
group by metric_name, parent_metric_name


Output:



Metric_Name Parent_Metrics_Name Total_Time_Value Count
AGENT_WAIT_TIME TOTAL_WAIT_TIME 0 0
AUDIT_FILE_WRITE_WAIT_TIME TOTAL_WAIT_TIME 0 0
AUDIT_SUBSYSTEM_WAIT_TIME TOTAL_WAIT_TIME 0 0
CLIENT_IDLE_WAIT_TIME
12781606
DIAGLOG_WRITE_WAIT_TIME TOTAL_WAIT_TIME 452 211
DIRECT_READ_TIME TOTAL_WAIT_TIME 13351 6066
DIRECT_WRITE_TIME TOTAL_WAIT_TIME 635 90
FCM_MESSAGE_RECV_WAIT_TIME FCM_RECV_WAIT_TIME 65799 35803
FCM_MESSAGE_SEND_WAIT_TIME FCM_SEND_WAIT_TIME 0 0
FCM_RECV_WAIT_TIME TOTAL_WAIT_TIME 66430 35943
FCM_SEND_WAIT_TIME TOTAL_WAIT_TIME 0 0
FCM_TQ_RECV_WAIT_TIME FCM_RECV_WAIT_TIME 631 140
FCM_TQ_SEND_WAIT_TIME FCM_SEND_WAIT_TIME 0 0
IPC_RECV_WAIT_TIME TOTAL_WAIT_TIME 414 105880
IPC_SEND_WAIT_TIME TOTAL_WAIT_TIME 443 108850
LOCK_WAIT_TIME TOTAL_WAIT_TIME 0 0
LOG_BUFFER_WAIT_TIME TOTAL_WAIT_TIME 0 0
LOG_DISK_WAIT_TIME TOTAL_WAIT_TIME 48250 6832
POOL_READ_TIME TOTAL_WAIT_TIME 1398 990
POOL_WRITE_TIME TOTAL_WAIT_TIME 1233 147
TCPIP_RECV_WAIT_TIME TOTAL_WAIT_TIME 0 0
TCPIP_SEND_WAIT_TIME TOTAL_WAIT_TIME 0 0
TOTAL_COMMIT_PROC_TIME TOTAL_RQST_TIME 7344 11901
TOTAL_COMPILE_PROC_TIME TOTAL_RQST_TIME 13021 19386
TOTAL_IMPLICIT_COMPILE_PROC_TIME TOTAL_RQST_TIME 0 0
TOTAL_LOAD_PROC_TIME TOTAL_RQST_TIME 0 0
TOTAL_REORG_PROC_TIME TOTAL_RQST_TIME 0 0
TOTAL_ROLLBACK_PROC_TIME TOTAL_RQST_TIME 2485 19189
TOTAL_ROUTINE_USER_CODE_PROC_TIME TOTAL_RQST_TIME 23046 1247
TOTAL_RQST_TIME
504097 149909
TOTAL_RUNSTATS_PROC_TIME TOTAL_RQST_TIME 8968 61
TOTAL_SECTION_PROC_TIME TOTAL_RQST_TIME 21926 31140
TOTAL_SECTION_SORT_PROC_TIME TOTAL_SECTION_PROC_TIME 22 182
TOTAL_WAIT_TIME TOTAL_RQST_TIME 132649
WLM_QUEUE_TIME_TOTAL TOTAL_WAIT_TIME 0 0

Combining DB2 and OS Diagnostics Part 1

Often there is need to combine DB2 diagnostic data with OS. Here are few useful commands and utilities you can use on different platforms..



AIX HP-UX Solaris Linux
OS level /usr/sbin/oslevel /usr/bin/uname -a /usr/bin/uname -a /bin/uname -a
Software /usr/bin/lslpp -ah /usr/sbin/swlist -v /usr/bin/showrev -p /bin/rpm -qa
CPUs /usr/sbin/lsdev -C (grep proc) /usr/sbin/ioscan (grep processor) /usr/sbin/psrinfo -v /bin/dmesg (grep CPU)
Memory /usr/sbin/lsattr -El sys0 (grep realmem) /usr/samples/kernel/vmtune /usr/sbin/dmesg (grep Physical) /usr/sbin/prtconf (grep Memory) /bin/dmesg (grep Memory) /usr/bin/free
IPC resources Ipcs /usr/sbin/kmtune SAM GUI tool /usr/sbin/sysdef -i /etc/system (file) /usr/bin/ipcs -l
User limits ulimit -a /etc/security/limits (file) ulimit -a ulimit -a ulimit -a
User environment Set Set Set Set



Windows NT has a nice interface for obtaining system information:
winmsd /af


On Windows 2000® and XP®:
<OSdrv>:\Program Files\Common Files\Microsoft Shared\MsInfo\Msinfo32
/report
This command does not report on installed software. One way to obtain this information is
to export the registry (at least the HKEY_LOCAL_MACHINE\SOFTWARE branch) to a
file by using the regedit tool.


System error and message logs on UNIX®
The following table shows where the logs are located on different UNIX platforms.



AIX HP-UX Solaris Linux
Error / Message Logs /usr/bin/errpt -a /var/adm/syslog/syslog.log /usr/sbin/dmesg /var/log/messages* /var/adm/messages*







Wednesday, January 5, 2011

DB2 Health Advisor Service

This is the new feature incorporated in DB2 UDB, V9 onward. It's aimed at minimizing unplanned outages, as DBA's are busy and overworked.

Most important it's FREE  !!!
With just one command(db2has) you can have a detailed report about your system mailed to you.

Type in 60-70 characters and get a detailed report.
db2has -icn 123456 -systype "test" -email “email@somecompany.com" -send

Where: icn is customer number.
            email: Report is sent to the requested email address.
            systype: production, dev, test.
            -send: ftp to IBM support site.

For detail about command refer to online info center.
http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?topic=/com.ibm.db2.udb.admin.doc/doc/r0056894.htm

It checks Recommend SW Check (OS, Java, etc), Tuning Check (based on PMRs), End of Support, Resource Utilization, Analysis of db2diag.log , OS configuration and it has the Comparison Feature (Comparison to previous scan or comparison to similar system).

When you run this command, it collects information about OS, hardware, patches, DB2 configuration and workload, in the form of xml file db2has_hostname_timestamp.xml in the default location sqllib/db2hasdir.
sends it across to IBM, where analytics service create the report that eventually send to email address provided.

It's available with DB2 V9.1, fixpack 9, DB2 V9.5, fixpack 6, DB2 V9.7 fixpack 1.


For earlier fixpacks, utility can be downloaded:
https://www.ibm.com/services/forms/preLogin.do?source=swg-beta-db2hasvc