SAP Sybase ASE already contains powerful data access control features like granular permissions to enforce the separation of duties. The data encryption option will extend and enhance these features with sophisticated, flexible algorithms and methodologies.

Data Encryption with SAP Sybase ASE

SAP Sybase ASE first introduced data encryption on the column level in release 12.5. In data encryption terminology there are two phases that compose the process. Encryption will obfuscate the data with the help of an encryption key and store the data in the database. The same encryption key is used to read the obfuscated data from the database and decrypt the data back into its original value.

SAP Sybase ASE 15.7 uses Advanced Encryption Standard (AES) with 128, 192, or 256-bit encryption key sizes for its data encryption methodology. The longer the bit string, the more difficult it is for an unauthorized person to decrypt the data.

Data encryption in SAP Sybase ASE uses a symmetric encryption algorithm, which means that the same key is used for encryption and decryption. When you insert or update data in an encrypted column, SAP Sybase ASE transparently encrypts the data immediately before writing the row. The more complicated the encryption, the more CPU resources will be taken up by the encryption algorithm. This methodology is called Transparent Data Encryption (TDE), and is used by Oracle and SQL Server as well. Both Oracle and SQL Server implemented TDE in their data encryption features, using an encryption key to encrypt and decrypt data. The difference is in the way they do the implementation.

Data encryption in SAP Sybase ASE is governed by a system security officer (SSO) or a Key Custodian. Both roles need to have the sso_role granted to their login profiles and no other user, including database administrators, should have this role granted to them.

Data encryption is an internal affair with SAP Sybase ASE. The master encryption key can be stored in the database itself. SAP Sybase ASE features key encryption keys and key encryption values in its data encryption algorithm. The key encryption key (KEK), formerly known as system encryption key (SEK), now has four levels of security. The most secure option requires a password that is not saved in the database. The loss of this password will make it impossible to decrypt the data in any way whatsoever. As a best practice procedure, the Key Custodian should appoint a trusted user as the key recoverer and in case of a lost password; this recovery password can be used to recover master key access.

SAP Sybase ASE 15.7’s data encryption also allows you to encrypt and store passwords from other systems in the database by using a “Service” key. This includes passwords from CIS, LDAP, SSL and Rep Server.

Implementing Data Encryption

Generally, enabling SAP Sybase ASE data encryption consists of a few steps, but requires in-depth planning to ensure that no data loss occurs due to lost passwords.

The steps involved are:

1. The SSO (assigned sso_role with an admin login or a Key Custodian) enables column encryption with this command:sp_configure ‘enable encrypted columns’, 1
2. Depending on the protection method of your column encryption keys, you need to create a database-level master key or set the system encryption password. This is the key encryption key (KEK) that is being used to create the column encryption keys (CEK).
3. Create one or more column encryption keys (CEK). If additional passwords are assigned to encryption keys, data will be protected, even from database administrators.
4. Specify the columns in a table to be encrypted.
5. Grant decrypt permission to users who must see the data. You can choose a default plain text to mask data for unauthorized users.

Once these steps have been completed, existing applications can run against existing tables and columns, but now the data is securely protected against theft and misuse.

The control over the encryption process, when choosing a database-level master key, can be delegated into an application layer. There is no external protection mechanism to be considered, and the application retains full control. This is important to 3^rd party applications as the SAP Sybase ASE data encryption is a licensed option. If a client choses to implement data encryption later on, the application must be able to dynamically enable encryption for its data without manual intervention. If one or more encryption keys are located outside the database, the application loses control over the encryption admin process.

SAP Sybase ASE gives applications the authority to take control of data encryption with database-level master keys and SQL statements based column encryption.

Encrypting Data

SAP Sybase ASE is able to encrypt most of its data types. For obvious reasons, encrypting text and image data types is not possible. You also cannot encrypt computed columns or specify encrypted columns in the partition_clause of a table. Other than that, you’re free to apply data encryption to as many columns as you prefer.

In the simple encryption sample below, you can see how to create two encryption keys, and encrypt two columns with two different keys. This is a new table encryption sample.
create encryption key default_key as default for AES
create encryption key cc_key
create table emp_table (ssn char(15) encrypt, ename char(50),...)
create table customer (creditcard char(20) encrypt with cc_key, cc_name char(50),...)
The ssn column will be encrypted with the default key, and the creditcard column will be encrypted with the cc_key. The column encryption keys (CEK) are being created using the key encryption keys (KEK).  If a password is required you will be prompted to enter the password to create the CEK.

To apply encryption to a column in an existing table, use the alter table / modify command to modify the column. You will use either the default key or a named encryption created in the current database or another database. The exception for this encryption is that you cannot modify a column if there is a trigger in place, or the column is a key in a clustered or placement index. In this case, the trigger / index has to be dropped, the column encrypted, and the trigger / index re-created.

Understanding Decryption

One important part of encrypting data in a column is to ensure that the values can be decrypted. The decrypt permission can be granted either to a role or individually to a user on specific tables. It is advisable to combine select and decrypt permissions at the same time to a user or role. Be careful when granting DML (update, insert and delete) permissions without the decrypt permission to a user or role. This can cause unintended consequences and potentially lead to data loss. Encrypted data can be overwritten with trash so that it is impossible to decrypt the data. In this case, encryption constraints or planning carefully for data access permissions are the best solutions to prevent this scenario.

Data Encryption vs. Data Masking

One of the main distinctions between data encryption and data masking is that encryption has to be reversible and a decryption method has to ensure that encrypted data can be successfully decrypted into the correct value.  Data masking applies data scrambling or obfuscation methods to render sensitive data unusable. One of the most famous data masking examples is displaying only the last four digits of a social security number. All other digits have been replaced by the letter x.

Data masking is divided into static and dynamic masking. Obfuscation either happens in real time when accessing the data, or the data has been store in an obfuscated form into the database. Static data masking is not possible in production databases; it is commonly used to protect sensitive data in test and development environments.

As an added bonus with data encryption, basic data masking can be achieved via the default value setting when encrypting data columns. If a user has not been granted decrypt permission, a default value will be displayed instead.

Data Encryption Consequences

When implementing data encryption with SAP Sybase ASE, special attention to index strategies is required to offset negative performance impacts. That’s why it is important to choose the encrypted columns wisely. SAP Sybase ASE has built-in optimization indexing encrypted columns.  You can access data effectively via an index, but there are certain rules that need to be followed. The same holds true when encrypting constraints.

The biggest consequence when implementing data encryption is the potential loss of data if passwords are lost. SAP Sybase ASE offers four options to manage the key encryption keys (KEK), each one of them with a larger degree of protection. There is still the standard system encryption key (SEK) option available, but all other options require one or more passwords, and these passwords are not stored in the database. Keeping track of these passwords is one of the duties of the SSO or the Key Custodian.

Performance Impacts

Based on independent performance benchmarks, the impact of data encryption on performance largely depends on how many columns are being encrypted. There is a storage increase associated with data encryption, which gets easily overlooked in encryption rollout projects, but could create unintended bottlenecks.

Creating or re-creating indexes will take a performance hit. This is expected behavior; however SAP Sybase ASE implemented performance improvements in its latest version that will mitigate this performance impact.

If encrypted columns are part of a table join in a query, the largest performance impact will be noted, especially if non-encrypted columns are being joined with encrypted columns. Be sure to thoroughly check your encryption plan with the data model and the foreign key reference model.

Expanding Data Encryption beyond Data

SAP Sybase ASE provides strong data encryption methodologies for external login passwords and hidden text, using the AES-256 symmetric encryption algorithm. By using this option, you will be able to apply strong data encryption for external passwords to:

-          Replication Agents and replicated databases

-          CIS for remote descriptors and logins

-          The job scheduler agent

-          Real Time Messaging (RTMS)

-          Secure Socket Layers (SSL) and Lightweight Directory Access Protocol (LDAP) protecting SSL and LDAP access accounts. Passwords are being managed by the stored procedures sp_ldapadmin and sp_ssladmin, and can be secured.

Securing external passwords and hidden text requires the use of a service key. Service keys are 256-bit, persistent encryption keys and are stored in sysencryptkeys. Service keys are encrypted by either using a static key or the master key. Using the master key requires the password to decrypt the database-specific master key.

Conclusion

Encrypting data will protect data at rest from unauthorized access. In case of backup files getting stolen or privileged OS users gaining access to the database files, data encryption renders these security breach attempts useless. Data encryption is required to comply with certain government and industry regulations such as PCI, HIPPA, etc. SAP Sybase ASE implemented data encryption in a secure and flexible way that enables applications to take control over the data encryption process.

SAP Sybase ASE enables data encryption by storing the key encryption keys inside the database. It also added an additional security layer requiring granting special decrypt permission to users in order to make the data readable again. This effectively blocks database administrators from accessing sensitive data in the database and on the backup files.

Based on benchmark tests, data encryption adds very limited overhead to the data processing if used for selective columns in selective tables. The CPU overhead logically increases when encrypting more columns. But as a tradeoff for stellar security, this is a small price to pay.

About the Author

Peter Dobler is an accomplished IT database professional who delivers results by building sophisticated data systems, by implementing advanced application software, and by improving efficiencies and reducing costs. He founded Dobler Consulting, (www.doblerconsulting.com) a Tampa, Florida consulting firm that delivers implementation expertise for Oracle, SAP Sybase, and MS SQL Server in 2000. His 25 plus years in technology started in Basle, Switzerland in 1985. Peter uses his extensive experience to hone his talent as a proven resource for producing streamlined IT solutions. He currently engages in strategic alliances and special projects with SAP Sybase, the enterprise software and services company. He can be reached at: pdobler@doblerconsulting.com or 813-322-3240.

Read his Database Trends Blog (Behind the Scenes of Database Evolution) at: http://www.doblerconsulting.com/db-tech-trends, and his Technology Tips Blog (Step-by- step Instructions on Today’s Challenging Technology) at  http://www.doblerconsulting.com/tech-tips-tricks.

The post SAP Sybase ASE – Keeping Private Data Private appeared first on Dobler Consulting.

SAP Sybase ASE 15.7 ESD#2 has introduced a new security feature called Granular Permissions. These are grantable system privileges that allow you to enforce “separation of duties.” To better understand the role of data security and implementing Granular Permissions into an existing environment, it is important to see the bigger picture of data security tasks. The level of data security needs depend largely on the size and industry of your company. Industry and government regulations dictate your data security requirements.

Click Here to read more about Granular Permissions and how SAP Sybase ASE implements fine tuned separation of duties.

The post SAP Sybase ASE 15.7 ESD#2 – Separation of Duties appeared first on Dobler Consulting.

Besides the in-memory capabilities, SAP Sybase IQ’s new features range from row-level versioning, the portability of models with PMML, integration modes for Hadoop and MapReduce and new data protection features.

SAP Sybase IQ is further strengthen its best choice position in the traditional Data Warehouse solution market with its latest version by affordably powering and supporting extreme data analytics on commodity hardware for thousands of concurrent users.

Click here to read more about SAP Sybase IQ.

The post SAP Sybase IQ 16 Goes Real-Time appeared first on Dobler Consulting.

IDC interviewed 12 large organizations with over 12,000 employees who have RDBMS systems, who are in a variety of industries. Six of the companies are running databases from multiple vendors.
Here are some of their conclusions. (Note: These findings are for a five year total cost per 100 users and a 240 gigabyte database).

Sybase ASE 15.7 total costs for the 5 years are $339,773. Using Sybase saved each of these 12 companies $128,580 over other RBDMS. These savings were reflected in IT staffing, hardware, software licenses, and database downtimes.

The following features contributed to this large cost savings:

• Automated functions for DBA’s
• Self-management capabilities for databases to tune themselves
• Data compression to reduce storage footprint
• Large object blocks (LOBs) stored in-row, rather than as separate objects.
• In-place execution of operations instead of using temp storage

After describing Sybase’s ability to lower the total costs of operation, IDC suggests that DBA’s need to:

• Review periodically the total costs of ownership for their RDBMS
• Use a 5-year time frame to understand the impact of growth on servers, storage, and fees for their RDBMS
• Collect information about other RDBMS products to be able to make good comparisons
• Be open to database conversion for software applications. It may not be as difficult or risky as you think.

IDC is careful to note that database products are constantly evolving. They suggest Sybase and their competitors need to keep addressing customer requirements to remain cost-competitive.

See (Sybase IDC Calculating the true cost of RDBMS) for the full article.

Our conclusion is: IDC paints an excellent picture for considering Sybase 15.7. They have a solid grasp on what’s important when considering TOC for databases. We’ll find out if Sybase can keep their cost-savings momentum. Right now, their strategy for keeping database technology costs low is winning them many new customers.

The post Sybase Ranks as Number 1 in IDC White Paper for Lowest Database Costs Over 5 Years appeared first on Dobler Consulting.

This month’s newsletter describes 3 database technologies that offer new promise for improving performance: Sybase® ASE 15.7, Sybase® IQ 15.4, and Microsoft SQL Server® 2012 RC0.

Sybase® ASE.15.7

On September 13th this year, Sybase announced availability of Sybase® Adaptive Server Enterprise® (ASE) 15.7. This is exciting news for our database community. According to Carl Olofson, research vice president at IDC, “Sybase has made significant enhancements to optimize data storage and increase developer productivity in ASE 15.7.” “These new features . . . add to ASE’s already impressive total cost of ownership.”

ASE 15.7 (the most feature-rich release in their history), is offering many new capabilities. Some of the most important ones include:

• Analyzing Dynamic Parameters
• Compressing Data in Adaptive Server
• Creating Non-materialized, Non-null Columns
• Cursor changes
• ERP certification on Sybase ASE
• Expanded Variable-Length Rows
• Fully Recoverable DDL
• Large objects changes
• Like Pattern matching changes
• Nested select statement enhancements
• New Adaptive Server Kernel
• New security features
• Retaining monitoring data
• Select for update changes

For more detailed information, see Sybase’s full feature list at:
http://www.sybase.com/detail?id=1094783 .

Sybase® IQ 15.4

Sybase has also launched the new version of the Sybase IQ high performance column-based analytics database. Sybase IQ® 15.4 is addressing the new generation of Big Data. DBA’s and database developers can use this version to store large amounts of unstructured data, and web log information. Sybase tells us we’ll be able to use a “have it your way” approach.

Some new capabilities to consider are: a native MapReduce API, Predictive Model Markup Language (PMML) support, and integration with Hadoop. You will be able to federate a query between IQ and Hadoop. The new APIs will allow developers to implement in-database algorithms with greater performance results.

PMML capabilities will work with predictive models in industry-standard analytics tools, and then be able to automate their execution in Sybase IQ. These features can also be combined with text and multimedia analytics, using a range of techniques for analyzing big data.

Some other advantages with this new version are:

• New Java API and new extensions to the C++ APIs, allows implementation of high performance proprietary or certified ISV algorithms in-database.
• In-database analytics simulation environment eases development and testing.
• Compression of text data works well with other big data scenarios.
• Faster bulk loading of large data sets works through ODBC and JDBC interfaces

For more details on Sybase IQ 15.4 see Sybase’s what’s new datasheet at:

http://www.sybase.com/detail?id=1095620.

Microsoft announced SQL Server 2012 Release Candidate (RC0) last month, with its Cloud offerings.

There are three main SQL Server 2012 editions – Enterprise, BI (Business Intelligence), and Standard. Microsoft is still offering Web Developer and Express versions of SQL Server 2012.

At Dobler Consulting we’re seeing some exciting new features with this version:

Cloud Advantages

• SQL Server Data Tools
• Data-tier Application Component (DAC) parity and increased object support with SQL Azure
• SQL Azure DataSync
• Support for OData

Mission Critical Abilities

• SQL Server AlwaysOn
• ColumnStore Index
• User Defined Roles
• Distributed Replay

Breakthrough Insight

• Power View Web-based Data Visualizations
• BI Semantic Model
• Data Quality Services
• Migrate PowerPivot models into SSAS

Our conclusion . . . Sybase and Microsoft are offering us innovative and exciting new ways to manage our database systems and improve performance.

Download SQL Server 2012 RC0 via this link:

http://www.microsoft.com/download/en/details.aspx?id=28145

The post Database Technology For 2012 appeared first on Dobler Consulting.

The International Sybase User Group, an independent association that represents thousands of users of Sybase products in more than 60 countries around the world, is pleased to announce the appointment of Peter Dobler as ISUG Partner Director.

Speaking on the news of his recent appointment, Peter said, “Partners are a very important support structure for any member organization, especially for ISUG. A strong and active partner structure will provide extra benefits for the ISUG members and in return the partners will gain market presence and recognition. This is a win-win situation for both the ISUG partners and the ISUG members. Dedicating more resources and attention to the partner program will benefit everybody.”

Go to the ISUG website at http://www.isug.com

The post ISUG Community Guru Appointed Partner Director appeared first on Dobler Consulting.

Our meeting will begin with an opportunity to network with your peers, followed by some brief introductions and any business the group needs to conduct. Then we’ll move swiftly to the first of our technical presentations delivered by Sybase Rockstar, Dave Fish who will be LIVE in person!

PowerBuilder 12.5 has a lot of new features for both .NET and Win32 developers. In this session Dave will demonstrate the new .NET targets for PowerBuilder Assemblies, WCF Services, and Custom Visual User Objects (CVUOs) as .NET Assemblies. John will also demonstrate several new WPF DataWindow enhancements, and how PowerBuilder applications can call RESTful web services. Win32 developers will learn how they can update the look of their applications with new DataWindow enhancements as well as see new features for PowerBuilder Classic controls.

Up next will be Sue Dunnell, the PowerBuilder Product Manager who will present the roadmap for PowerBuilder with focus on the soon-to-be-released PowerBuidler 12.5 as well as the next release (tentatively 15) and beyond. As part of this session, Dave will give a sneak preview of enhancements planned for PowerBuilder 15, including the HTML5 DataWindow, and Dockable window support for Win32 PowerBuilder applications.

Afterwards we’ll have a round-table discussion about PowerBuiler. This session is aimed at all PowerBuilder users and is your chance to share your thoughts about the current direction for PowerBuilder and to give direct feedback to the PowerBuilder Product Manager on how Sybase can better position Sybase for the future.

Sue will also be available following this for individual one-on-one/small group sessions where you can sit down with her to discuss PowerBuilder needs that are specific to your business (and perhaps confidential). These are a great opportunity for your management to talk directly to the PowerBuilder product manager! You can register for one-on-sessions during the meeting registration process.

This event is free to all attendees.

When & Where

Thursday, August 11th 2011

3:30 Networking
4:00 Introductions, group business
4:15 PowerBuilder 12.5, PowerBuilder Classic – Dave Fish
5:45 PowerBuilder roadmap; v15 enhancements – Sue Dunnell / Dave Fish
6:30 PowerBuilder round-table
7:30 Individual sessions
8:00 Meeting concludes

Click here to RSVP

At: Dobler Consulting
8270 Woodland Center Blvd
Tampa, FL 33614

Click here for directions

Please consider to upgrade your ISUG membership. This will allow us to fly in high quality speakers to present in person and interact with you directly.

For our out of town friends:

The live meeting link will be provided through the official ISUG registration process. Please check the confirmation email for the live meeting link.

The post ISUG User Group Meeting August 11th appeared first on Dobler Consulting.

The topic I will be presenting is the highly charged theme of “Oracle to Sybase Migration”. This is a topic loaded with emotions and opinions. I think it is refreshing that Sybase is acknowledging this topic, as all the other vendors are doing for years. Let’s face it, prior to SAP most, if not all, Sybase migration projects where from Sybase to anywhere else. To me this is a huge showing of confidence from Sybase.

Here’s the official title and description of the session I will present at TechWave:

A Practitioner’s Guide to Successfully Migrate from Oracle to Sybase ASE (Parts 1 & 2)

CIO’s unanimously agree that reducing infrastructure costs is key to ensuring adequate budget investment to achieve top-line business growth.  While there are obviously many approaches to achieving this, one area gaining attention is migrating from the higher cost Oracle database platform to Sybase ASE,  Long known as the highly performant and resilient database powering much of Wall Street, its’ flexibility and low total cost of ownership, both in hardware and administrative costs, are making CIO’s take notice.  This 2-part session will explore this topic from two perspectives.  We will first dive into this topic from the perspective of the business’ "Program Manager" by identifying the major high-level tasks and milestones of a typical project plan as well as quantifying the level of effort and risk for each.  We will then explore this from an implementation perspective looking at some of the most critical and resource-intensive technical aspects of such projects to help you minimize the effort and risk of these types of career-enhancing projects.  If your company is considering its options for a competitive, yet cost-effective, database infrastructure, this is a session you can’t afford to miss.

Please join me at the most exciting TechWave in years. For more information go to:
http://www.sybase.com/techwave

See you there,
Peter

The post Meet Me in Las Vegas appeared first on Dobler Consulting.

Bottom-line, the more members ISUG can attract the more influential it gets and this transfers directly to you. With your feedback and product enhancement request, ISUG is in a position to make your word heard by Sybase.

As long time member of ISUG and advocate for its cause, becoming a partner with my company Dobler Consulting was a logical step. Starting in October of last year, Dobler Consulting is hosting the Tampa Bay ISUG meetings and since the re-launch we got into a rhythm of having a meeting every other month. The next meeting is scheduled August 11th and we will have an afternoon session with PowerBuilder and its roadmap, followed by an evening session on Sybase 15.7. Details will be released as soon as the schedule and agenda are confirmed.

Please visit the ISUG website at http://www.isug.com and if you’re not a member, sign up today.

Take care,
Peter

The post Dobler Consulting Is Now an ISUG Partner appeared first on Dobler Consulting.

The solution to these data base challenges was to split a single table into multiple partitions that could be accessed independently and still maintain the data integrity of a single table. A second solution was the introduction of row level locking to drastically reduce data insert contention.

Still, more data distribution challenges call for other solutions. Partitions were slowly introduced with Sybase ASE 11 to reduce contention and support parallel processing. However, it was not until ASE 15 that partitions jumped back into the spotlight.

Benefits Using Semantic Partitions

There are several advantages to using semantic partitions. For example, you’ll have reduced maintenance time and more predictable maintenance tasks that will reduce “just in case” maintenance, because you can perform maintenance tasks on just one or more partitions, instead of the whole table (which many DBAs practice).

A second benefit is for applications that are dealing with date and time based data, like sales data, telecommunication information, bank transactions or patient insurance information. They will all experience a dramatic performance gain with range partitioning.

Reduced maintenance windows and improved query performance with range partitioning also boosts productivity on more than one level.

Two of the semantic partition types offer specific gains:

• Hash partitioning doesn’t provide the flexibility of range partitioning. Yet, its strength is in spreading the data over as many partitions as possible to provide a performance boost to non-time based queries.
• Rolling range partitions benefit from archive data placements onto cheaper hardware.  This means that data can “age”, and older static data rows will reside on partitions that can be moved from” tier 1” storage to” tier 3” storage. Cost savings between “tier 1” and “tier 3” storage is significant. The more data, you can move to “tier 3” storage, the bigger the savings. Rolling range partitions are not a function of Sybase’s semantic partitions, but a technique to effectively manage date driven data. The key is to create date range partitions without a MAX limit. This will manage the data growth and future data distribution into the range partitions.

Reduce Maintenance Time Using Semantic Partitions

By using the new semantic partitions in Sybase ASE 15, DBAs are learning that having increased operational scalability is a genuine reality, as a result of reduced maintenance time.

Recent internal benchmarking at a client installation tracked time reductions for update statistics tasks y a factor of 25 – 30. The 25 – 30 factor is absolutely correct . . .  reductions for update statistic tasks by double digit factors, not percentages! These reductions are significant, and allow DBAs to reclaim their maintenance window. In other words, by using semantic partitions, a 3 hour update statistics job completes in 6 minutes.

Improve Availability and Maintenance Using Semantic Partitions

By reducing the maintenance windows, application availability automatically increases.  As mentioned earlier, reducing maintenance jobs from 3 hours down to 6 minutes will repurpose at least 2 hours for application use, which translates in 2 hours of added productivity for applications and users.

Semantic partitions also allow DBAs to be more selective and pro-active in the choice of maintenance tasks. In large systems, it is not uncommon for maintenance to be spread over every day of the week to guarantee a complete weekly cycle The time window allowed for maintenance is too short to complete the task in one step. Having the ability to complete this maintenance in the short time frame is a huge advantage. By using the semantic partitions, previously occupied system resources quickly become available again to applications.

What Type of Partitions Does Sybase Use?

There are different types of partitions that can be used to take control over the data distribution of a single table. Each table can maintain its own partition type.

Round-Robin Partitioning

This was the first partition type introduced in Sybase ASE 11.0.3. The data is evenly distributed amongst the total number of partitions available. There are drawbacks with this partition type. The most severe is that you cannot dynamically add a new partition to an already partitioned table. The only way to add a new partition is to unpartition the original table, and then repartition the table.

Another drawback to Round-Robin partitioning is that partitions need to be rebalanced manually and often. Each time you rebalance, you have to drop and recreate the clustered index for this table. Most installations used a very low number of partitions, no more than 10, because the rebalancing was labor- intensive, complex, and time-consuming.

This is the only partition type available prior ASE 15.

Semantic Partitioning Options

Sybase introduced semantic partitions in its ASE 15 version. Semantic partitioning uses the data values to decide which partition the data resides in vs. the session id.

Sybase has 3 types of semantic partitioning: range, hash and list.

Range Partitioning

The Range partitioning is the most flexible and adaptable partition type in ASE 15. Some of the advantages to using range partitioning are:

• Reduces maintenance time by eliminating the need to run maintenance tasks on older static data rows.
• Implements rolling, alternating or migrating partitioning schemes to constantly archive older data rows.

Range partitioning offers, by far, the most features and flexibility to manage time-period aware data. In today’s data explosion, the bulk of the data is growing in this area. An excellent use for range partitioning is financial transaction information, with time-period aware data.

Hash Partitioning

Like the Round-Robin partitioning, hash partitioning is distributed evenly over the available partitions. The advantage to hash partitioning is that no rebalancing is necessary. The Hash partition type is most effective with data that does not have a life cycle, like sales records, or data that does not require any form of range queries. This partition type is most likely used on primary keys.

The drawback is you have the same flexibility limitations for hash partitioning you had with Round-Robin partitioning.

List Partitioning

This partition type allows you to dynamically add new partitions without unpartitioning and repartitioning the table. Like hash partitioning, the list partitioning offers the best performance on queries with exact matches. This means “column = constant” works best with this partition type.

A natural fit for list partitioning data is data organized by geographic region or separating branch data by store location. The key to a successful list partitioning implementation is low cardinality. Based on performance benchmarks cardinality of less than 10 is most effective with list partitioning.

New Index Types

With the introduction of semantic partitions, Sybase also introduced a new type of indexes. Global and local indexes are being used in tandem to effectively manage the data on partitions. Local indexes are used to create an index on a single partition. Local indexes have no knowledge about data outside the partition they are managing. Thus, creating a unique key on a local index is not possible. The advantage to using local indexes is that the index tree has few levels, which reduces the amount of I/O. For example, if you can reduce local index levels from 6 or 7 to 3 or 4, your I/O reduction will be significant. You will have less contention on the index root page also. 

Global indexes spawn all partitions for the entire table, and index all rows in the entire table. To enforce primary key uniqueness, a global index is necessary. As a rule of thumb, use global indexes only if absolutely necessary. A performance benchmark study compared a drop partition task performed on a partitioned table with a global index vs. local indexes. The tasks completed in seconds when using local indexes, but ran for a couple of hours to keep the global index in sync.

Semantic Partitions Offer Full Control over Data Placement

When partitions were first introduced, the primary goal was to reduce contention during insert statements by creating several last data pages. Up to this point, clustered indexes were the only method to distribute inserts.

Prior to the semantic partitions in Sybase ASE 15, only Round Robin partitioning was available as a partition type. Round Robin partitioning solved the insert contention problem, and improved parallel SQL execution, but left another serious problem. Round robin partitioning gave DBAs no control over the data placement within the partitions.

With the introduction of semantic partitions, users now have full control over data placement within individual partitions of a table. This opens the door to very finely tuned maintenance tasks that slash the time required to complete these activities. Prior to semantic partitions, all maintenance tasks were at the table level. At that time, maintenance on all partitions of a table needed to be completed to finish the maintenance. Partitions now can be used to spawn several database processes in parallel, speeding up execution significantly.

Where Semantic Partitions Are Not a Good Fit

Semantic partitions can improve almost any data distribution situation. In many cases, there are performance improvements with queries as well. It is fair to say that adding semantic partitions offers improvement opportunities with almost no side effects.

However, applying semantic partitions to all tables is not a free ride for DBAs.  Administrative overhead still exists. DBAs need to keep an eye on partitions to ensure optimal performance to the applications.

Internal analysis showed that partitions have no effect on contention elimination when used on data row locking scheme tables. This means that row level locking provides enough separation to deal with table contention.

Before using range or hash partitioning, DBAs need to consider:

• Although the range partitioning is the most powerful semantic partition type, studies show there were no performance improvements past the date range search arguments on time period based data.
• Applying hash partitioning to time based data is not a good idea. It can have a negative impact on both the query performance and the maintenance time allocated to maintain this constellation.

Sybase provides the tools and metrics to determine the best use of semantic partitions. The two considerations mentioned above will help DBAs make the best decisions for their situation. DBA’s can use Sybase’s expertise, along with deciding what their own best performance needs are.  

What’s Next

Sybase is planning to introduce several new features on semantic partitions in the near future. All of them will further enhance the performance and the maintainability of partitioned data.
The most important future enhancements are:

• Unique keys on local indexes
• Merge partitions
• Split partitions

Sybase is committed to provide the best tools supporting DBAs to tame the data volume explosions and providing the highest level of database uptime to end-users and applications.

Conclusion

At first glance, semantic partitions may only seem to be valuable during maintenance tasks, but a closer look under the covers reveals that there are more benefits than meets the eye.

Hash partitioning and even Round-Robin partitioning over many partitions can dramatically improve performance over primary key data access. Hash partitioning thrives with equality search arguments in queries. List partitioning over low cardinality data, like geographic location distribution, can open new maintenance opportunities for DBAs maintaining a global data and user community. Performing data maintenance on partitions for geographic regions that are separated by global time zones allows DBAs to maintain optimal performance on databases that do not allow for downtime.

Rolling range partitions provide the capability to automatically “roll off” archive data, based on date ranges, onto cheaper disk hardware. This will save money and does not require additional maintenance. With range partitioning, maintenance tasks can be applied to individual partitions without the need to perform maintenance on all partitions of a table. With the datachange() function, DBAs can determine which partitions need maintenance. This allows for surgical precision in maintenance efforts and dramatically reduces the time frame to execute these tasks.

About the Author

Peter Dobler is an accomplished IT database professional who “makes a difference” by improving efficiencies and reducing costs for small and medium-sized businesses.  He founded Dobler Consulting, (www.doblerconsulting.com) a Tampa, Florida consulting firm that delivers implementation expertise for Oracle, Sybase, and MS SQL Server in 2000. His 25 plus years in technology started in Basle, Switzerland in 1985. Peter uses his extensive experience to hone his talent as a proven resource for producing streamlined IT solutions. He currently engages in strategic alliances and special projects with Sybase, the enterprise software and services company. He can be reached at: mailto:pdobler@doblerconsulting.com or 813-322-3240.

Read his Database Trends Blog (Behind the Scenes of Database Evolution) at: http://www.peterdobler.com, and his Technology Tips Blog (Step-by- step Instructions on Today’s Challenging Technology) at http://www.techtipsntrick.com.
Other Articles of Interest:
“Sybase ASE 15.5 — The Need for Speed”, Database Journal, www.databasejournal.com, May 2010

Reviews on Peter Dobler’s articles

“IMDB said to be the New Frontier of Database Architecture,” http://www.dbajobsandcareers.com, Database Blog, Review, June 1, 2010
“Performance Enhancement and the In-Memory DBMS Opportunity”, International Sybase User’s Group,  http://www.isug.com/common/Index.html, March, 2010

The post Sybase ASE 15 – Semantic Partitions to the Rescue appeared first on Dobler Consulting.