Transactions and Versioning

Conflicting transactions

To see how IQ works with transactions and versioning open two sessions to IQ and run the following set of commands:

Session-1: create table t1(a int)
Session-1: begin tran
Session-2: create table t2(a int)
Session-1: insert into t2 values(1)

The following error is then raised:

ASA Error -1000011: Transaction 156593 attempted to access an object created by transaction 156608.
-- (db_txnInfo.cxx 690)
Sybase error code=21, SQLState=”QDA11”

In the IQ message file the error is also reported:

I. 01/07 14:55:58. 0000000563 Exception Thrown from db_txnInfo.cxx:690, Err# 0, tid 478 origtid 478
I. 01/07 14:55:58. 0000000563    O/S Err#: 0, ErrID: 1025 (db_catalogException); SQLCode: -1000011, SQLState: 'QDA11', Severity: 14
I. 01/07 14:55:58. 0000000563 [20671]: Transaction 156593 attempted to access an object created by transaction 156608. 
-- (db_txnInfo.cxx 690)

This error happened because IQ works with table level versioning. Every transaction in IQ gets a number and that transaction can not deal with data from transactions that have started later (have a higher number). In the example, table t2 is created within a transaction with a higher number than the transaction in session 1 that tried to insert a row into table t2. Under normal situations you will not often hit this problem, but it can happen when you run multiple sessions in IQ updating data at the same time.

Coding around conflicting transactions

To make an application resilient for this you can use error trapping, as in the following example. Again, open two sessions to IQ and run the following set of commands:

Session-1: create table t1(a int);
Session-1: begin tran;
Session-2: create table t2(a int);

Then run a command in Session-1 that traps for the error and give a message. Run the following piece of code in one batch.

begin
  declare tran_error exception for SQLSTATE 'QDA11';
  insert into t2 values(1);
  message 'Everyting ok, no conflicting transaction' to client;
  exception
    when tran_error then message 'You hit a conflicting transaction' to client;
    when others then resignal;
end;

Locking

In Sybase IQ only a single session can write to a particular table at any given time. There can be multiple update sessions going at the same time, but each session should update a different table. When you try to break this rule the following error message is raised:

Msg 8405, Level 21, State 0:
SQL Anywhere Error -210: User 'Joe' has the row in 'myTable' locked

In the IQ message log the following is raised:

I. 02/16 16:29:53. 0000000898 Txn 114153 0 114153
I. 02/16 16:29:53. 0000000898    sqlcode -210
I. 02/16 16:29:53. 0000000898    string_id 2169
I. 02/16 16:29:53. 0000000898    _sqlstate 67792968
I. 02/16 16:29:53. 0000000898    odbc30state 67108865
I. 02/16 16:29:53. 0000000898    sybcode 8405
I. 02/16 16:29:53. 0000000898    severity 21
I. 02/16 16:29:53. 0000000898    odbc20state 67108865
I. 02/16 16:29:53. 0000000898 Exception Thrown from db_catalog.cxx:717, Err# 0, tid 1979 origtid 1979
I. 02/16 16:29:53. 0000000898    O/S Err#: 0, ErrID: 5122 (st_sacbexception); SQLCode: 0, SQLState: '00000', Severity: 10
I. 02/16 16:29:53. 0000000898 [2000]: 
I. 02/16 16:29:53. 0000000898 Rbck
I. 02/16 16:29:53. 0000000898 PostRbck
I. 02/16 16:30:10. 0000000950 Txn 114155 0 114155
I. 02/16 16:30:10. 0000000950 Cmt 114156
I. 02/16 16:30:10. 0000000950 PostCmt 0

In Sybase IQ 15 the “lock table” command can now be used to request a lock on a table and when the lock cannot be granted just wait until it can. An example of this is:

lock table <table-name> in write mode wait "00:10:00"

he commands request a lock for a table and when it is granted no other processes can write to the same table. When the lock cannot be granted the lock command will wait 10 minutes as specified with “00:10:00″.

By preceding each insert/update/delete or load table command by a “lock table” the error message can be easily prevented when the application requires it.

The post Sybase IQ Versioning and Locks appeared first on Dobler Consulting.

Below you will find a script that does a basic attempt to reverse engineer the statements for the “create database” and “create dbspace” commands. The script is not complete with regard to all the possible options for these statements. Feel free to make adjustments or mail the author when you miss something.

NB: Written specifically for IQ 15 !

begin
declare @db_file        long varchar;
declare @iq_file        long varchar;
declare @msg_file       long varchar;
declare @temp_file      long varchar;
declare @block_size     unsigned int;
declare @chunk_size     unsigned int;
declare @reserve_size   rowid;
declare @user_dbspace   long varchar;
declare @user_dbfile_name long varchar;
declare @user_file_name long varchar;
declare @counter        int;

declare local temporary table #statement(
        seq             int             not null default autoincrement,
        statement_txt   long varchar    null);

declare end_of_cursor exception for SQLSTATE '02000';

declare user_dbspaces cursor for
        select  dbspace_name
                from sysdbspace
                where   store_type      = 2
                and     dbspace_name    not in ('IQ_SYSTEM_MAIN', 'IQ_SYSTEM_TEMP', 'IQ_SYSTEM_MSG');

declare user_dbfiles cursor for
        select  f.dbfile_name,
                f.file_name
                from    sysdbfile f,
                        sysdbspace d
                where   d.store_type    = 2
                and     d.dbspace_name  = @user_dbspace
                and     d.dbspace_id    = f.dbspace_id
                order   by dbfile_id;

select  file_name into @db_file
        from    sysdbfile
        where   dbspace_id      = 0;

select  file_name into @iq_file
        from    sysdbfile
        where   dbfile_name     = 'IQ_SYSTEM_MAIN';

select  file_name into @msg_file
        from    sysdbfile
        where   dbfile_name     = 'IQ_SYSTEM_MSG';

select  file_name into @temp_file
        from    sysdbfile
        where   dbfile_name     = 'IQ_SYSTEM_TEMP';

select  block_size, chunk_size into @block_size, @chunk_size
        from    sysiqinfo;

select  reserve_size into @reserve_size
        from    sysiqdbfile
        where   dbfile_id       = 16384;

insert  into #statement(statement_txt)
        select  'create database  + @db_file + ' +
        ' IQ PATH  + @iq_file + ' +
        ' IQ PAGE SIZE ' + convert(varchar,@block_size * @chunk_size) +
        ' IQ RESERVE ' + convert(varchar,convert(int,(@reserve_size * 1024)/ @block_size)) +
        ' BLOCK SIZE ' + convert(varchar,@block_size) +
        ' MESSAGE PATH  + @msg_file + ' +
        ' TEMPORARY PATH  + @temp_file + ;' as "statement";

insert  into #statement(statement_txt)
        select  'alter dbspace ' + d1.dbfile_name +
                ' add file ' + d2.dbfile_name + '  + d2.file_name + ;' as "statement"
                from    sysdbfile d1,
                        sysdbfile d2
                where   d1.dbfile_name  in ('IQ_SYSTEM_MAIN', 'IQ_SYSTEM_TEMP')
                and     d1.dbfile_id    = d1.dbspace_id
                and     d2.dbspace_id   = d1.dbspace_id
                and     d2.dbfile_id    != d2.dbspace_id;

open    user_dbspaces;

user_dbspace_loop:
LOOP
        fetch   next user_dbspaces into @user_dbspace;
        if      SQLSTATE = end_of_cursor then
                leave user_dbspace_loop;
        end if;

        insert  into #statement(statement_txt)
                select  'create dbspace ' + @user_dbspace + ' using ';

        set     @counter = 0;

        open    user_dbfiles;

        user_dbfiles_loop:
        LOOP
                fetch   next user_dbfiles into @user_dbfile_name, @user_file_name;
                if      SQLSTATE = end_of_cursor then
                leave   user_dbfiles_loop;
        end if;

        if      @counter = 0 then
                insert  into #statement(statement_txt)
                        select  'file ' + @user_dbfile_name + ' + @user_file_name + ' as statement;

                set     @counter = @counter + 1;
        else
                insert  into #statement(statement_txt)
                        select  ',file ' + @user_dbfile_name + '  + @user_file_name + ' as statement;
        end if;

        end loop user_dbfiles_loop;
        close user_dbfiles;

        insert  into #statement(statement_txt)
                select 'iq store;' as "statement";
end loop user_dbspace_loop;

close   user_dbspaces;

insert  into #statement(statement_txt)
        select  'alter dbspace ' + d.dbspace_name +
                ' striping ' + case sb.striping_on when 'T' then 'on' else 'off' end +
                ' stripesizeKb ' + convert(varchar,stripe_size_kb) + ';' as "statement"
                from    sysiqdbspace sb,
                        sysdbspace d
                where   d.dbspace_id    = sb.dbspace_id
                and     d.store_type    = 2
                and     d.dbspace_name  != 'IQ_SYSTEM_MSG';

select  statement_txt
        from    #statement
        order   by seq;
end;

The post Reverse Engineer Sybase IQ dbspaces Without an ER Tool! appeared first on Dobler Consulting.

What You Can Do to Influence DQP Scalability and Effectiveness.

There are various server and database operations that affect parallelism and performance of a query. After you have maximized Indexes using the Sybase IQ Index Advisor, there are several database options which will exploit Parallel Query processing which I listed below. I have used the below “options” in a variety of test cases and are all dependent on numerous variables which are to numerous to mention. If you meet the above prerequisites, try them and see how you may benefit from their usefulness. Some are just Best Practices which should be included as default and some are data dependent, Hardware related and other factors.

• Max_Query_Parallelism: this database option sets an upper bound which limits how parallel the optimizer will permit query operators, such as joins, GROUP BY and ORDER BY. The default value is 64. Systems with more than 64 CPU cores often benefit from a large value — up to the total number of CPU cores on the system to a maximum of 512.

• Minimize_Storage: set this database option to ‘on’ prior to loading data into tables, or utilize IQ_UNIQUE on column definitions. FP(1), FP(2) and FP(3) indexes that use lookup tables will be created instead of flat FP indexes. These take up less space and decrease I/O (although FP(3) indexes consume a lot of memory, so utilize them judiciously). This database option is almost always set as a default.

 • Force_No_Scroll_Cursors: if you do not need backwards scrolling cursors, set this database option to ‘on’ to reduce temporary storage requirements. Almost always used!

• Max_IQ_Threads_Per_Connection: controls the number of threads for each connection. With large systems, you may see some performance benefit by increasing this value.

• Max_IQ_Threads_Per_Team: controls the number of threads allocated to perform a single operation (such as a LIKE predicate on a column). With large systems, you may see some performance benefit by increasing this value.

• Max_Hash_Rows: set this database option to 2.5 million for each 4GB RAM on the host. For example, set it to 40 million on a 64GB system. This will encourage the query optimizer to utilize hash based join and group by algorithms, which scale better. However, there is a caveat here: with very large hash tables, it is possible for performance to regress when distributed due to the time required to flush hash tables on one node and reconstitute them on another. DQP will attempt to compensate for this and not distribute hash based operators when the hash table becomes prohibitively large, even if memory can accommodate it.

• -iqgovern: this server option specified the number of concurrent queries on a particular server. By specifying the –iqgovern switch, you can help IQ maintain throughput by giving queries adequate resources to commit quickly. The default value is (2 x number of CPUs) + 10. For sites with large numbers of active connections, you might want to set this value lower.

• -iqtc: this server option sets the temp cache size. Temp cache is used by both the local and shared temporary stores. DQP must utilize IQ_SHARED_TEMP in order to do its processing, and therefore requires adequate temp cache. You may want to allocate more memory to it than main cache for DQP workloads. There are a couple of DQP specific database options that are offered as well. There to lengthy to discuss in this session.

• MPX_Work_Unit_Timeout: when a worker node does not complete processing of its query fragment within the mpx_work_unit_timeout value, the work is passed back to the leader to retry. If you find that timeouts are occurring and adversely affecting the performance of DQP, you can increase the timeout value to allow a worker to complete. Generally, though, you are unlikely to hit a timeout issue unless you have some other underlying problem.

• DQP_Enabled: this is an option you can set for a database connection. If DQP is occurring, but you are not seeing benefits from it, you can turn it off.

Sizing Shared Temporary Storage

An adequate amount of shared temporary space on fast storage hardware is critical for the performance of distributed queries. While it is difficult to calculate in advance how much shared temporary storage you will need for a distributed query, there are some trends that have been observed:

• Use of shared temporary space can vary widely among nodes in the PlexQ grid as they are executing a distributed query

• The amount of shared temporary space used does not correlate with the scalability of the query. Queries that do not scale well may use as much or more shared temporary space as queries that do scale well.

• Queries that use more temporary cache/space when running on a single node will tend to use more shared temporary space when running distributed, but there is not an obvious multiplier that can be derived.

The post Optimizing and exploiting Sybase IQ Distributed Query Processing (DPQ Tips N Hints) appeared first on Dobler Consulting.

Coordinator Node

Each multiplex configuration requires a coordinator node. When you convert an existing simplex server to multiplex, it becomes the coordinator node.

The coordinator node:

• Runs read-only and read-write operations against shared IQ objects.
• Manages IQ main dbspaces.
• Manipulates local data in SQL Anywhere system and user tables.

Coordinates all read-write operations on shared IQ objects, including:

• Shared IQ table locking
• Shared IQ storage management
• Providing global transaction IDs for read-write transactions involving shared IQ objects
• Maintaining the global catalog
• Controls catalog synchronization for secondary servers
• Performs schema changes on shared IQ store objects
• Performs schema changes on SQL Anywhere store objects
• Maintains and cleans up object versions

Secondary NODES.. ie (reader/writer)

One or more secondary nodes may participate in a Sybase IQ multiplex configuration. One secondary node acts as a designated failover node, the first choice node to assume the coordinator role if the current coordinator is unable to continue.

The number of secondary nodes supported depends on the license purchased, as follows:

• Demo/Trial Edition: Unlimited secondary nodes
• Small Business Edition: None (multiplex not allowed)
• Single Application Server Edition: One secondary node
• Enterprise Edition: Unlimited secondary nodes (license needed for each)

Secondary nodes:

• Can be either read-only nodes (reader nodes) or read-write nodes (writer nodes).

Writer nodes:

• Can run read-only and read-write operations against shared IQ objects.
• Can manipulate local data in temporary and SA base tables.

Reader nodes:

• Can run read-only operations against shared IQ objects.
• Can manipulate local data in temporary and SA base tables.

The post Sybase IQ Multiplex Nodes and Server Overview appeared first on Dobler Consulting.

Why on Earth does IQ only allow you to create indexes with parallel option? Ask Sybase!

BEGIN PARALLEL
CREATE  HG index idx_1_HG ON <table> (column_1);
CREATE LF index idx_2_LF ON <table> (column_2);
CREATE HG index idx_3_HG ON <table> (column_3);
...... on and on
END PARALLEL


I have a  300 million row table joined to 1.2 million row table and have some records I want to delete and was wondering why it’s taking longer than it should . How can I coax some more speed out of my deletes? Auh, IQ Optimization options fills this need.  Specifying predicates on columns that have HG indexes greatly improves costing. In order for the HG costing to pick an algorithm other than large delete, it must be able to determine the number of distinct values (groups) affected by deletions.  1st their has to be a HG index on the column in the predicate which is usually the case. IQ chooses three algorithms to process deletes.

set option HG_DELETE_METHOD = 1;  <------------'Small' delete' for a few distinct  rows or 1 row
set option HG_DELETE_METHOD = 2; <------------ 'Large delete'  100000+ rows
set option HG_DELETE_METHOD = 3; <------------' Mid delete' 100 to 10000 rows


The post Sybase IQ Index Tips and Tricks appeared first on Dobler Consulting.

This overview is for IQ 15.x only, so lets talk about the FP index in some detail.

IQ has 9 types of indexes but only three are highly used and a couple are considered useless in 15x. In this Blog, I will only discuss the most frequently used which is the FP and explain it’s usages.

—FAST PROJECTION (FP)

When creating a table, IQ creates this type for free (By default) but you can optimize them accordingly.  There are 4 types of FP indexes, 1 byte, 2 byte, 3 byte and flat. You want to avoid going flat because that means the index is not optimized. These types are dependent on 2 factors

– The cardinality of the data

– The value of IQ unique given in the ‘create table ‘ statement or if the, set option  MINIMIZE_STORAGE=’ON’ which is    like declaring IQ UNIQUE (255) on every column. Many times you will see this option hint if you use the INDEX_ADVISOR. If you see this recommendation when you analyze the query output html, you know your flat (No FP optimization) I’ll explain

The space  used to store the index is dependent on the type of the FP index used and the data type being stored.

A flat index stores the data as it’s being loaded with no storage optimization. For example, a char(20) will store 20 bytes of storage per value. As mentioned before, we want to use the 1,2 or 3 byte FP index and logic says the lower the byte the faster.

The 1 and 2 byte FP indexes are known as optimized FP indexes because they read faster from disk, use less disk space and are used by the query optimizer to obtain information about the data being stored in the column.

The 3 byte is the same as 1 and 2 byte except it holds between 65k and 16 million unique values. Again, It’s easier to sort through zeros and ones than a bunch of characters.

In Order to create an Optimized index you must either specify a value for IQ UNIQUE in a create table statement or set the MINIMIZE_STORAGE database option to ON. MINIMIZE_STORAGE = Guess, compression which another feature IQ uses to speed things up. If you intend on setting the MINIMIZE_STORAGE option, you must do this prior to creating the table where you want the IQ UNIQUE value to defaulted to.

Example;

CREATE TABLE order_entry
(
order_id        int   IDENTITY,
customer_id     int   IQ UNIQUE (255)   NOT NULL,
sales_id        int   IQ UNIQUE (10000)   NOT NULL,
product_id      int   IQ UNIQUE (5000)   NOT NULL
)
SET OPTION MINIMIZE_STORAGE='ON';

CREATE TABLE statement

In a flat FP index a column of raw data is compressed but their is no storage optimization used. 1,2 and 3 byte indexes convert via look up tables.. ) zeros and ones.  Minus pictures I hope I explained  how IQ utilizes FP indexes and the DBA control over their optimization.

The post Sybase IQ 15.x Fast Projection Index Overview and Practical Usage appeared first on Dobler Consulting.

Sybase IQ can create CIS connections to ASE, for example to access ASE data through an IQ-to-ASE proxy table.
This can be done in two ways: through JDBC and ODBC. JDBC is easier to set up, but ODBC tends to be significantly faster. In addition, the JDBC connectivity class has been deprecated (If you try to use CLASS ‘asejdbc’, You will receive a message box error via ISQL ”asejdbc has been deprecated”)   in IQ 15.4 on Linux.

With IQ running on Linux, as is this case, setting up ODBC connections to ASE can be tricky, to say the least. What it doesn’t help is that the IQ documentation isn’t exactly a concise example of clarity on this point, either. Below is what I found out the hard way about setting up IQ-to-ASE ODBC connectivity on Linux

IQ versions covered
The information about IQ-to-ASE ODBC connections below applies to IQ 15.4. As for OpenClient, I’ve used OCS 15.4, though it should also work for OpenClient 15.0 + (wasn’t tested). If you run into problems, (go to www.sybase.com -> Support -> EBFs/Maintenance -> Software Developer Kit).

IQ-to-ASE ODBC connections – basic syntax
To set up an ODBC connection from IQ to ASE, you need to use the following IQ statement:

create server MY_ASE_ODBC class 'aseodbc' using 'MY_ASE'

Precisely what to specify in the USING ‘…’ clause depends on which of these two variations is chosen:

1. Specify the ODBC connection attributes directly in the IQ create server statement;
2. Use an ODBC Data Source Name (DSN), which is defined outside IQ and holds the ODBC connection attributes.

We’ll look at both of these.

In the examples below, I’m assuming we’re on Linux and we’re trying to connect from IQ to an ASE server named ‘MY_ASE’ running on port 5000 on host ‘ASE_15′.
An IQ proxy table is then created to an ASE table named my_tab (owned by dbo) in databases my_db.

The IQ-to-ASE ODBC driver – three vital setup steps
But first, and before going into either of the ODBC variations above, you need to get the following three things right:

1. You need to have the correct ODBC driver library installed: the filename of the driver lib is libsybdrvodb.so.
   However, this driver may not be bundled with your IQ installation. If it is not, the driver is included with ASE as well as with the OpenClient connectivity libraries (downloadable from www.sybase.com -> Support -> EBFs/Maintenance -> Software Developer Kit).
   You must make sure you have this driver installed. It is OK, and perhaps even best, to install it in its own $SYBASE directory (it does not need to be in the IQ installation directory tree) . I copied the file and pasted it in several directories so I knew the ENV path would recognize it.
2. The second important thing is that you need the 64-bit version of the ODBC driver if your IQ server is 64-bit, and the 32-bit driver if you’re still running 32-bit IQ.
   It is vital to get this right. On the IQ side, you can find the IQ server’s bitness (if that’s considered a word) with a simple select @@version or iqsrv15 -v Sybase
IQ/15.4.0.6567/111107/P/GA/Enterprise Linux64 - x86_64 - 2.6.9-67.0.4.ELsmp/64bit/2011-11-07 00:40:24
   For the ODBC driver, the pathname tells you whether it’s 32- or 64-bit: the location is usually $SYBASE/DataAccess/ODBC/lib/libsybdrvodb.so or /opt/sybase/ASE_15_5/DataAccess64/ODBC/lib
3. The third essential requirement is that the pathname of the directory containing the ODBC driver file is in the IQ server’s $LD_LIBRARY_PATH environment variable (or equivalent on other platforms). This means you need to run the following before starting the IQ server:
   export LD_LIBRARY_PATH=
/opt/sybase/ASE_15_5/DataAccess64/ODBC/lib:$LD_LIBRARY_PATH

With these points covered, you can now start your IQ server…

IQ-to-ASE ODBC connection, without DSN
Now, let’s look at defining an ODBC connection from IQ to ASE with declaring the ODBC attributes directly in the IQ create server statement (i.e. without using an ODBC DSN).

• – create remote server mapping through ODBC
  create server MY_ASE_ODBC_1 class 'aseodbc'
  using ‘Driver=libsybdrvodb.so;Server=bigbox;Port=5000;Database=my_db;PacketSize=16384;EnableServerPacketSize=0′Important:The Driver= clause must specify the ODBC driver’s filename (libsybdrvodb.so). It may also specify the full pathname to the ODBC driver file, but the driver’s directory pathname must still be included in the IQ server’s $LD_LIBRARY_PATH.Note that this also configures -optionally- a network packet size of 16KB; the ASE server must be able to handle this size, or the connection will fail (don’t ask me why EnableServerPacketSize=0 is needed — the PacketSize keyword was ignored without it).
  Also note that the contents of the using ‘…’ clause are not checked at this stage; any errors will only be raised once IQ tries to make an actual connection.

• – create external login for the IQ ‘dba’ user
  – let’s assume it maps to the ASE ‘sa’ user; adjust as needed
  CREATE EXTERNLOGIN dba to MY_ASE_ODBC_1
REMOTE  login sa IDENTIFIED BY sql;Important: You must create an external login: even though it looks as if you can specify the remote login name+password directly with using ‘…;UID=sa;Password=sql; the specified password is ignored (for some reason, with using ‘…;UserID=sa;…’, the specified username is ignored as well).

• – test the connection; if there is any error in the setup, error messages will be raised now:
  forward to MY_ASE_ODBC_1 { select @@servername, db_name(), @@version };

• – if the connection works, create the proxy table:
  create existing table ase_proxy_tab at 'MY_ASE_ODBC_1.my_db.dbo.my_tab';select * from ase_proxy_tab;

The post Creating IQ Connections to ASE to Import Data appeared first on Dobler Consulting.

After installing CentOS you need to install these additional rpm packages.

• libXext
• libXtst

You can find the latest download versions for these packages at http://rpmfind.net

Failing to install these packages before running setup.bin will result in these errors:

Graphical installers are not supported by the VM 

./setup.bin
Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...
Launching installer...
Graphical installers are not supported by the VM. The console mode will be used instead...
Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)
Stack Trace:
java.lang.UnsatisfiedLinkError: /tmp/install.dir.27720/Linux/resource/jre/lib/i386/xawt/libmawt.so: libXext.so.6: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at sun.security.action.LoadLibraryAction.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.awt.NativeLibLoader.loadLibraries(Unknown Source)
at sun.awt.DebugHelper.<clinit>(Unknown Source)
at java.awt.Component.<clinit>(Unknown Source)
at com.zerog.ia.installer.util.BidiUtilImpl.setDefaultLocale(DashoA10*..)
at ZeroGbd.a(DashoA10*..)
at com.zerog.ia.installer.LifeCycleManager.s(DashoA10*..)
at com.zerog.ia.installer.LifeCycleManager.b(DashoA10*..)
at com.zerog.ia.installer.LifeCycleManager.a(DashoA10*..)
at com.zerog.ia.installer.Main.main(DashoA10*..)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.zerog.lax.LAX.launch(DashoA10*..)
at com.zerog.lax.LAX.main(DashoA10*..)
This Application has Unexpectedly Quit: Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)


Respectively….

./setup.bin
Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...
Launching installer...
Graphical installers are not supported by the VM. The console mode will be used instead...
Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)
Stack Trace:
java.lang.UnsatisfiedLinkError: /tmp/install.dir.32049/Linux/resource/jre/lib/i386/xawt/libmawt.so: libXtst.so.6: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at sun.security.action.LoadLibraryAction.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.awt.NativeLibLoader.loadLibraries(Unknown Source)
at sun.awt.DebugHelper.<clinit>(Unknown Source)
at java.awt.Component.<clinit>(Unknown Source)
at com.zerog.ia.installer.util.BidiUtilImpl.setDefaultLocale(DashoA10*..)
at ZeroGbd.a(DashoA10*..)
at com.zerog.ia.installer.LifeCycleManager.s(DashoA10*..)
at com.zerog.ia.installer.LifeCycleManager.b(DashoA10*..)
at com.zerog.ia.installer.LifeCycleManager.a(DashoA10*..)
at com.zerog.ia.installer.Main.main(DashoA10*..)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.zerog.lax.LAX.launch(DashoA10*..)
at com.zerog.lax.LAX.main(DashoA10*..)
This Application has Unexpectedly Quit: Invocation of this Java Application has caused an InvocationTargetException. This application will now exit. (LAX)

The post Installation Errors with Sybase IQ 15.4 on CentOS appeared first on Dobler Consulting.

Shared Installation

With the new 15.5 version, you have to option to install the software either as shared installation or private installation. If you use the shared installation, you need to have access to a shared filesystem that is accessible from every node in the cluster. Although the shared installation is more convenient, it presents other risks like a single point of failure if the shared filesystem goes offline.

Private Installation

The private installation provides a dependency separation between nodes for added stability and protection. The private installation installs the Sybase software on each node, and does not require a shared filesystem. You need to maintain a strict file structure and placement discipline, because every node must access the software identically.

Pre-Upgrade Steps

Once you completed the prerequisites, you need to shift your focus to preparing the databases, and the database server.

1. If you are upgrading Adaptive Server, the previously installed version of the server must be running. If you are upgrading Backup Server, Historical Server, Monitor Server, or XP Server, those servers must not be running.
2. Stored procedure text in the syscomments table is required for the upgrade. If you deleted the text, you must add it back again.

   Note: As a best practice: if you don’t want to display the text, hide it by using the sp_hide_text stored procedure instead of deleting it.

3. Resolve reserved words using quoted identifiers. This is a simple check by installing the upgrade package and then executing sp_checkreswords.

   Caution: This step is simple enough, but if omitted, can lead to serious issues during the upgrade process.

4. Perform some standard tasks that apply to any database server upgrade.
   1. Verify users are logged off.
   2. Check for database integrity. Run DBCC commands to complete this step.
   3. Back up the databases. As mentioned before, this will be your lifeline in case of a failed upgrade.
   4. Ensure that master is the default database for the “sa” user.
5. Prepare the database and devices for upgrade by following these steps:
   1. Disable auditing
   2. Disable Job Scheduler by ensuring the “enable Job Scheduler” is off.
   3. Archive auditing data and truncate auditing tables.
   4. Disable disk mirroring.

      Note: Sybase ASE Cluster Edition 15.5 does not support disk mirroring. This is important if you used the disk mirror approach to move your local database devices to the SAN. Please make sure that all device mirrors have been disabled.

   5. Verify that your $SYBASE environment variable points to the location of the new Adaptive Server software files you just installed.

Manual Upgrade of an Existing ASE Server

Your upgrade approach will be completely different based on the various upgrade options. I want to focus on the manual upgrade from a non-cluster ASE server to the ASE Cluster Edition 15.5.

For the full details of the manual upgrade, please review the Sybase ASE Cluster Edition Upgrade manual. The summary of the steps is:

1. In order for Sybase ASE Cluster Edition to work and communicate, the unified agent must be running on each node of the cluster.

   Note: Now is a good time to get into the habit of starting, and verifying the unified agent before starting any database server.

   Start the Unified Agent:

   $SYBASE/UAF-2_5/bin/uafstartup.sh &

2. Start your existing Sybase ASE server. Change the $SYBASE and $SYBASE_ASE variables to reflect the new location of the software. This process must be repeated when a restart of the existing Sybase ASE server is required.

    

3. Execute the $SYBASE/$SYBASE_ASE/upgrade/preupgrade command from the new software location to prepare your server for the upgrade. If there are errors reported, correct them and restart your existing Sybase ASE server. Repeat this step until no errors are displayed.

    

4. Check your existing Sybase ASE databases for new “reserved words” by installing and executing the sp_checkreswords stored procedure. Correct any errors prior to continuing the upgrade process.

   Caution: Omitting this step can lead to serious problems during the upgrade process.

    

5. One important part of installing a Sybase ASE Cluster Edition server is the requirement to have at least 2 network connections; 3 connections are even better. The additional network connections are needed for the server to interconnect via a primary private network and an optional secondary private network. In our example, we are using 2 private interconnects. Plus the public network access.

    

6. After shutting down the old server, you need to proceed with the cluster preparation. The first step is creation of a new cluster input file that describes your cluster environment. The first instance of the cluster must be the old server name. For this example, the filename mycluster.inp has been chosen.

   In addition, you need to have the network interconnect working. This is the back bone connection between the cluster nodes.

   Here is an example of the mycluster.inp file, based on a shared installation:

   #all input files must begin with a comment
   
   [cluster]
   name = mycluster
   max instances = 2
   master device = /dev/raw/raw1
   interfaces path = /sybase/
   traceflags =
   primary protocol = udp
   secondary protocol = udp
   
   [management nodes]
   hostname = syb1
   hostname = syb2
   
   [instance]
   id = 1
   name = syb1
   node = syb1
   primary address = syb1-ppriv
   primary port start = 38456
   secondary address = syb1-spriv
   secondary port start = 38466
   errorlog = /sybase/ASE-15_0/install/syb1.log
   interfaces path = /sybase/
   traceflags =
   additional run parameters =
   
   [instance]
   id = 2
   name = syb2
   node = syb2
   primary address = syb2-ppriv
   primary port start = 38556
   secondary address = syb2-spriv
   secondary port start = 38566
   errorlog = /sybase/ASE-15_0/install/syb2.log
   interfaces path = /sybase/
   traceflags =
   additional run parameters = 

7. Create the quorum device with the input file create in step 6. This is the core of the share disk cluster.

   Start the new instance with the old master device:

   $SYBASE/$SYBASE_ASE/bin/dataserver
   --instance=server_name
   --cluster-input=mycluster.inp
   --quorum-dev=/dev/raw/raw102
   --buildquorum
   -M$SYBASE

    

8. You’re ready to run the upgrade utility. instance_name is the first instance in your cluster that has the same name as the server from which you are upgrading:

   $SYBASE/$SYBASE_ASE/upgrade/upgrade  -S instance_name –Ppassword
     

9. Create a tempdb for each instance in the cluster.

   Note: This step is important. Without having the global temporary database for the second node in place, the cluster won’t start.

   1>create system temporary database tempdb1 for instance syb1 on tempdb1 = 100
   2>go
   1>create system temporary database tempdb2 for instance syb2 on tempdb2 = 100
   2>go

   tempdb1 and tempdb2 are new raw devices on the SAN, accessible by both nodes. The size of the tempdb is arbitrary.

10. Restart the cluster with the quorum device in the run file:

     $SYBASE/$SYBASE_ASE/bin/dataserver 
     --instance=server_name
     --quorum-dev=/dev/raw/raw102
     -M$SYBASE

     

11. Finish the upgrade with running a few scripts as described in the Sybase ASE Cluster Edition installation manual.

    Note: This is an abbreviated version of the entire install procedure, but it demonstrates how straight-forward the upgrade actually is. As always, please review the Sybase ASE Cluster Edition Upgrade manual for details, as the configurations may be different for your environment.

Once you upgraded your existing Sybase ASE server, you can add new nodes and convert your non-clustered ASE server into a multi node cluster with ease.

Conclusion

Upgrading your existing Sybase ASE server to Sybase ASE Cluster Edition is pretty straight forward. Especially if your ASE Server is on release 15.x. Keep in mind that with the 15.x release, a new query optimizer was introduced, and extra steps to mitigate possible performance degradation have to be exercised. Once you upgraded your ASE server, you now have access to new tools and methods to address availability and scalability challenges.

In my humble opinion, this is possibly the easiest upgrade path from a non-cluster database system to a shared disk cluster. Sybase ASE Cluster Edition brings your organization better database resources, uses less hardware, and strengthens your computer applications.

The post Upgrade Sybase ASE to Sybase ASE Cluster Edition – Step by Step appeared first on Dobler Consulting.

The 5 parts of this series are:

• Part 1: System Requirements and Test Setup Concept
• Part 2: Setup of the VMWare Server and the Linux Servers
• Part 3: Install of the Sybase Software
• Part 4: Setup and test the Sybase Cluster
• Part 5: Add an additional node to the cluster

The instructions designed that non-Sybase DBA can perform these tasks. Sybase DBA will find that the steps are very similar to your standard ASE installation.

Please click here to read about how to setup and test the Sybase Cluster.

Add an Additional Node to the Sybase Cluster

Preparation

The process adding a node to the cluster can be executed when the cluster is active. The steps are very simple and non-intrusive.

Note: As a pre-requisite the steps to setup the Linux environment for the original nodes must be followed before actually adding a node to the Sybase ASE cluster. The new node should already be configured with all network adapters, all the disks should have be mounted with right permissions etc.. Since the /sybase filesystem is a NFS share no other extra steps are needed for this preparation step. Simply follow the steps in the installation description. The only new raw filesystem that will be added is the temp device for the tempdb on the new node. This will be the new raw device raw 9 and must be mounted to all nodes.

The new /etc/raw looks like this and has to be the same in all nodes (asece1,asece2,asece3):

sybase@asece3:~> cat /etc/raw
# /etc/raw
#
# sample configuration to bind raw devices
# to block devices
#
# The format of this file is:
# raw:
#
# example:
# ---------
# raw1:hdb1
#
# this means: bind /dev/raw/raw1 to /dev/hdb1
#
# ...
raw1:sdb1
raw2:sdc1
raw3:sdd1
raw4:sde1
raw5:sdf1
raw6:sdg1
raw7:sdh1
raw8:sdi1
raw9:sdj1
sybase@asece3:~>

Please make sure that the /dev/raw/raw9 is owned by the sybase user.

The new /etc/hosts looks like this:
asece1:~ # cat /etc/hosts
#
# hosts This file describes a number of hostname-to-address
# mappings for the TCP/IP subsystem. It is mostly
# used at boot time, when no name servers are running.
# On small systems, this file can be used instead of a
# "named" name server.
# Syntax:
#
# IP-Address Full-Qualified-Hostname Short-Hostname
#
127.0.0.1 localhost
# special IPv6 addresses
::1 localhost ipv6-localhost ipv6-loopback
fe00::0 ipv6-localnet
ff00::0 ipv6-mcastprefix
ff02::1 ipv6-allnodes
ff02::2 ipv6-allrouters
ff02::3 ipv6-allhosts
# Public IP Addresses
192.168.1.210 asecenfs.localhost.org asecenfs
192.168.1.211 asece1.localhost.org asece1
192.168.1.212 asece2.localhost.org asece2
192.168.1.213 asece3.localhost.org asece3
# Rpimary Private Network
192.168.159.211 asece1-ppriv.localhost.org asece1-ppriv
192.168.159.212 asece2-ppriv.localhost.org asece2-ppriv
192.168.159.213 asece3-ppriv.localhost.org asece3-ppriv
# Secondary Private Network
192.168.207.211 asece1-spriv.localhost.org asece1-spriv
192.168.207.212 asece2-spriv.localhost.org asece2-spriv
192.168.207.213 asece3-spriv.localhost.org asece3-spriv
asece1:~ #

This is pretty much all the prep work that needs to be done to add a node to an existing Sybase ASE cluster.

Installation Steps

1. Increase the number of max instances parameter if not set to include an additional instance
The cluster must be shutdown to execute this command.
sybase@asece1:~> sybcluster -Uuafadmin -P -C mycluster -F "asece1,asece2"
> connect to mycluster
mycluster> shutdown cluster
Are you sure you want to shutdown the cluster? (Y or N): [ N ] y
INFO - Shutdown of cluster mycluster has completed successfully.
mycluster>set cluster maxInst 3
The value has been changed.
Would you like to recalculate the primary and secondary network ports? (Y or N): [ N ] Y
Enter the starting port number: [ 15100 ]
Recalculated port range: 1 Primary ase1 15100 to 15114
Recalculated port range: 2 Primary ase2 15115 to 15129
Recalculated port range: 1 Secondary ase1 15146 to 15160
Recalculated port range: 2 Secondary ase2 15161 to 15175
Should these port numbers be applied to the cluster? (Y or N): [ N ] Y
The values have been changed.

2. Start the unified agent on the new node
sybase@asece3>$SYBASE_UA/bin/uafstartup.sh &
3. Deploy the cluster configuration to the new node.
sybase@asece1:~> sybcluster -Uuafadmin -P -C mycluster -F "asece1,asece2"
>deploy plugin agent "asece3"
Enter the name of the cluster: mycluster
Verifying the supplied agent specifications...
1) asece3.localhost.org 9999 2.5.0 Linux
Enter the number representing the cluster node : [ 1 ] 1
Enter the full path to the quorum disk: /dev/raw/raw2
Enter the SYBASE home directory: [ /sybase ]
Enter the environment shell script path: [ /sybase/SYBASE.sh ]
Enter the ASE home directory: [ /sybase/ASE-15_0 ]
Deploying the cluster management agent plugin...
Agent plugin deployed successfully.
>exit


4. Add the new node/instance to the cluster
Before adding an instance to the cluster, the tempdb device must exist. Create the mycluster3_tempdb device on /dev/raw/raw9 either through Sybase Central or isql. Once created, continue with the procedure.
SQL Command:
USE master
go
disk init name='mycluster3_tempdb', physname='/dev/raw/raw9', vdevno=7, size=512000, cntrltype=0, dsync=true, directio=false
go
Note: The new instance is down by default and you start it manually…

mycluster> start instance ase3

sybase@asece1:~> sybcluster -Uuafadmin -P -C mycluster -F "asece1,asece2,asece3"
> connect to mycluster
mycluster>start cluster
.
.
.
.
INFO - 02:00:00000:00062:2008/12/23 08:37:01.75 kernel Sequence table svrid=2, tblcol=0, tblindex=1, count=0.
INFO - 02:00:00000:00062:2008/12/23 08:37:01.75 kernel instance 2 eventdone.
INFO - 02:00:00000:00002:2008/12/23 08:37:01.83 server ASE's default unicode sort order is 'binary'.
INFO - 02:00:00000:00002:2008/12/23 08:37:01.83 server ASE's default sort order is:
INFO - 02:00:00000:00002:2008/12/23 08:37:01.83 server 'bin_iso_1' (ID = 50)
INFO - 02:00:00000:00002:2008/12/23 08:37:01.83 server on top of default character set:
INFO - 02:00:00000:00002:2008/12/23 08:37:01.83 server 'iso_1' (ID = 1).

mycluster>add instance ase3
Verifying the supplied agent specifications...
1) asece1.localhost.org 9999 2.5.0 Linux
2) asece2.localhost.org 9999 2.5.0 Linux
3) ce3.localhost.org 9999 2.5.0 Linux
Enter the number representing the cluster node where ase3 will reside: [ 3 ] 3
--------------------------------------------------------
Instance Id 1; ase1 uses transport tcp on port 19786.
Instance Id 2; ase2 uses transport tcp on port 19786.
Enter the interface file query port number for instance ase3: 19786
Backup Server is configured for the cluster. Do you want to configure it for this instance too? [ Y ] Y
Enter the Backup Server port number for node "mycluster": 19799
Enter the primary protocol address for ase3: [ asece3.localhost.org ] asece3-ppriv
Enter the secondary protocol address for ase3: [ asece3.localhost.org ] asece3-spriv
--------------------------------------------------------
Calculating default ports. Please wait...
Currently defined protocols specifications by Instance
Instance Id 1; Name: ase1; Primary 15100 - 15114; Secondary 15146 - 15160
Instance Id 2; Name: ase2; Primary 15115 - 15129; Secondary 15161 - 15175
Enter the primary protocol starting port for instance ase3: [ 15130 ]
Enter the secondary protocol starting port for instance ase3: [ 15176 ]
--------------------------------------------------------
Device: master Used: 70 Mb Size 180 Mb
Path: /dev/raw/raw1
Device: tapedump1 Used: 0 Mb Size 0 Mb
Path: /dev/nst0
Device: tapedump2 Used: 0 Mb Size 625 Mb
Path: /dev/nst1
Device: sysprocsdev Used: 135 Mb Size 180 Mb
Path: /dev/raw/raw4
Device: systemdbdev Used: 12 Mb Size 80 Mb
Path: /dev/raw/raw5
Device: mycluster1_tempdb Used: 900 Mb Size 1,000 Mb
Path: /dev/raw/raw7
Device: mycluster2_tempdb Used: 900 Mb Size 1,000 Mb
Path: /dev/raw/raw8
Device: sybmgmtdev Used: 180 Mb Size 180 Mb
Path: /dev/raw/raw6
Device: data Used: 2,208 Mb Size 10,000 Mb
Path: /dev/raw/raw3
Device: mycluster3_tempdb Used: 0 Mb Size 1,000 Mb
Path: /dev/raw/raw9
--------------- Local System Temporary Database ---------
The Local System Temporary Database Device contains a database for each instance in the cluster.
Enter the LST device name: mycluster3_tempdb
Note: The device can be an existing one if there are enough spaces to it....
Enter the LST database name: [ mycluster_tdb_3 ]
Enter the LST database size (MB): [ 1000 ] 900
--------------------------------------------------------
Would you like to save this configuration information in a file? [ Y ] Y
Enter the name of the file to save the cluster creation information: [ /sybase/mycluster_ase3.xml ]
--------------------------------------------------------
Add the instance now? [ Y ]
INFO - Creating the Cluster Agent plugin on node asece3.localhost.org using agent: asece3.localhost.org:9999
A cluster agent plugin at asece3.localhost.org:9999 is already managing a cluster by the name mycluster.
Should this agent plugin be reused for the cluster: [ N ] Y
INFO - The Cluster Agent Plugin on agent asece3.localhost.org:9999 will be reused.
Adding the new instance...
INFO - ase3: Creating the Local System Temporary database ase3_tempdb on ase3_tempdb of size 900M.
The addition of the instance ase3 has completed.

sybc> show cluster status
INFO - Listening for the cluster heartbeat. This may take a minute. Please wait... (mycluster::AseProbe:434)
Id Name Node State Heartbeat
-- ---- ------------------ ----- ---------
1 ase1 asece1.localhost.org Up Yes
2 ase2 asece2.localhost.org Up Yes
3 ase3 asece3.localhost.org Down No
-- ---- ------------------ ----- ---------

This is all it takes to add an additional node to an existing Sybase cluster. It is simple, straight forward and can be implemented in minutes. The only downside is that the cluster has to be down in order to deploy a new node agent. If this could be done while the cluster is up and running that would be as close to complete dynamic horizontal deployment as it gets.

During my tests I repeated this step several times to see how easy and quickly it is to recover from a failed node expansion and it could be easier than this. Simply shutdown down the agent on the new node, remove the files in

$SYBASE_UA/nodes/asece3/plugins/mycluster

and repeat the steps above. It worked every time without a hitch. Every DBA will appreciate the simplicity.

Conclusion:
I hope this 5 part step-by-step mini course provided you with all the information needed to create your very own Sybase ASE 15 CE cluster environment.

Thank you for your interest and please stay tuned for more step-by-step instructions on other technologies.

The post Sybase ASE 15 Cluster Edition Step-By-Step Installation – Part 5 appeared first on Dobler Consulting.