Scale and Data Warehouse Performance
We know that scale is an important factor in the performance of a data warehouse.
A large FACT table may contain hundreds of millions of rows, and a full-table scan against this table could take days to complete.
What Does a Data Warehouse Need?
Since the earliest days of Decision Support Systems (DSS) of the 1960s, database professionals have recognized that internal processing for data warehouse applications is very different from Online Transaction Processing Systems (OLTP).
Data warehouse applications tend to be very I/O intensive, as the database reads trillions of bytes of information. Data warehouse systems require specialized servers that can support the typical processing that we see in data warehouses.
Most data warehouses are bi-modal and run batch process when new data is loaded, indexed, and summarized. The server must have on-demand CPU and RAM resources, and the database
management system must be able to dynamically reconfigure its resources to accommodate these shifts in processing.
While RAM costs continue to fall ever year, the speed of RAM access is constrained by silicon technology and has not improved in the past three decades.
Evolution Of The Data Warehouse
While the use of databases as a vehicle for complex data analysis is new, the need to perform complex data analysis has been with us for centuries.
Answering “what-if” questions, simulating the introduction of a new product, or determining the most profitable products are all legitimate business needs,
and the advent of the data warehouse did not herald the first time that computers had been used to solve these types of tasks.
In fact, computers have been used to solve complex types of data analysis problems since their earliest commercial inception in the 1950s.
Essentially, the nature of the questions that data warehouses solve has not changed in four decades. The summarization of numbers, the aggregation of associated data, and data variance analysis is nothing new.
Evolution Of Data Management Platforms
Regardless of the sophistication of a database manager, it remains true that all databases are constructed from simple data structures such as linked lists,
B-trees, and hashed files. In reviewing the building blocks of database systems, it is possible to gain a historical perspective on the evolution of
databases, and to remember the past, so that we are not condemned to repeat it.
The following pages are a historical review of database evolution, showing each of the enhancements that were introduced with each new architecture. It
is also important to review the problems inherent to each database architecture. As you will see, there are striking similarities between object-oriented
databases and earlier database architectures. By understanding the historical reason that object-oriented databases have evolved into their present form, we
can gain insight into the future trend and directions of databases.
Today's Oracle professional must be an expert in database performance and tuning, database
administration, data warehousing, using OLAP and spatial data, and other areas.
These robust features of Oracle present unique challenges to anyone who must use Oracle technology to deliver solutions to complex data-oriented challenges.
Oracle, the world's leading database management system, provides a vast wealth of features and options.
The Oracle market is filled with new possibilities as Oracle introduces the
management of objects, data warehouses, and Web-enabled applications; Oracle professionals everywhere are struggling to understand how to exploit these new features.