{"id":34,"date":"2023-01-23T17:11:16","date_gmt":"2023-01-23T17:11:16","guid":{"rendered":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/chapter\/module-8-chapter-notes\/"},"modified":"2023-03-24T14:17:52","modified_gmt":"2023-03-24T14:17:52","slug":"module-8-chapter-notes","status":"publish","type":"chapter","link":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/chapter\/module-8-chapter-notes\/","title":{"raw":"Module 8 Chapter Notes","rendered":"Module 8 Chapter Notes"},"content":{"raw":"<h2>Chapter 11 Notes for Module 8<\/h2>\n<p class=\"page-title\"><strong>What is SQL performance tuning?<\/strong><\/p>\nSQL performance tuning describes a process \u2013 on the client side \u2013 that will generate an SQL query to return the correct answer in the least amount of time, using the minimum amount of resources at the server end.\n<p class=\"page-title\"><strong>What is database performance tuning?<\/strong><\/p>\nDBMS performance tuning describes a process \u2013 on the server side \u2013 that will properly configure the DBMS environment to respond to clients\u2019 requests in the fastest way possible, while making optimum use of existing resources.\n<p class=\"page-title\"><strong>What is the focus of most performance tuning activities, and why does that focus exist?<\/strong><\/p>\nMost performance-tuning activities focus on minimizing the number of I\/O operations, because the I\/O operations are much slower than reading data from the data cache.\n<p class=\"page-title\">What are database statistics, and why are they important?<\/p>\nThe term\u00a0<i>database statistics<\/i>\u00a0refers to a number of measurements gathered by the DBMS to describe a snapshot of the database objects\u2019 characteristics. The DBMS gathers statistics about objects such as tables, indexes, and available resources -- such as number of processors used, processor speed, temporary space available, and so on. Such statistics are used to make critical decisions about improving the query processing efficiency.\n<p class=\"page-title\"><strong>How are database statistics obtained?<\/strong><\/p>\nDatabase statistics can be gathered manually by the DBA or automatically by the DBMS. For example, many DBMS vendors support the SQL\u2019s ANALYZE command to gather statistics. In addition, many vendors have their own routines to gather statistics. For example, IBM\u2019s DB2 uses the RUNSTATS procedure, while Microsoft\u2019s SQL Server uses the UPDATE STATISTICS procedure and provides the Auto-Update and Auto-Create Statistics options in its initialization parameters.\n<p class=\"page-title\"><strong>What database statistics measurements are typical of tables, indexes, and resources?<\/strong><\/p>\nFor tables, typical measurements include the number of rows, the number of disk blocks used, row length, the number of columns in each row, the number of distinct values in each column, the maximum value in each column, the minimum value in each column, what columns have indexes, and so on.\n\nFor indexes, typical measurements include the number and name of columns in the index key, the number of key values in the index, the number of distinct key values in the index key, histogram of key values in an index, etc.\n\nFor resources, typical measurements include the logical and physical disk block size, the location and size of data files, the number of extends per data file, and so on.\n<p class=\"page-title\"><strong>How is the processing of SQL DDL statements (such as CREATE TABLE) different from the processing required by DML statements?<\/strong><\/p>\nA DDL statement actually updates the data dictionary tables or system catalog, while a DML statement (SELECT, INSERT, UPDATE and DELETE) mostly manipulates end user data.\n<p class=\"page-title\"><strong>What are the three phases that the DBMS processes queries in and what is accomplished in each phase?<\/strong><\/p>\nThe three phases are:\n<ol>\n \t<li><i>Parsing<\/i>. The DBMS parses the SQL query and chooses the most efficient access\/execution plan.<\/li>\n \t<li><i>Execution<\/i>. The DBMS executes the SQL query using the chosen execution plan.<\/li>\n \t<li><i>Fetching<\/i>. The DBMS fetches the data and sends the result set back to the client.<\/li>\n<\/ol>\nParsing involves breaking the query into smaller units and transforming the original SQL query into a slightly different version of the original SQL code -- but one that is \u201cfully equivalent\u201d and more efficient.\u00a0<i>Fully equivalent<\/i>\u00a0means that the optimized query results are always the same as the original query. More efficient means that the optimized query will, almost always, execute faster than the original query. (Note that we say\u00a0<i>almost<\/i>\u00a0always because many factors affect the performance of a database. These factors include the network, the client\u2019s computer resources, and even other queries running concurrently in the same database.)\n\nAfter the parsing and execution phases are completed, all rows that match the specified condition(s) have been retrieved, sorted, grouped, and\/or \u2013 if required \u2013 aggregated. During the fetching phase, the rows of the resulting query result set are returned to the client. During this phase, the DBMS may use temporary table space to store temporary data.\n<p class=\"page-title\"><strong>If indexes are so important, why not index every column in every table?<\/strong><\/p>\nIndexing every column in every table will tax the DBMS too much in terms of index-maintenance processing, especially if the table has many attributes, many rows, and\/or requires many inserts, updates, and\/or deletes.\n\nOne measure to determine the need for an index is the data\u00a0<i>sparsity<\/i>\u00a0of the column you want to index. Data sparsity refers to the number of different values a column could possibly have. For example, a STU_SEX column in a STUDENT table can have only two possible values, \u201cM\u201d or \u201cF\u201d; therefore this column is said to have low sparsity. In contrast, the STU_DOB column that stores the student date of birth can have many different date values; therefore, this column is said to have high sparsity. Knowing the sparsity helps you decide whether or not the use of an index is appropriate. For example, when you perform a search in a column with low sparsity, you are very likely to read a high percentage of the table rows anyway; therefore index processing may be unnecessary work.\n<p class=\"page-title\"><strong>What is the difference between a rule-based optimizer and a cost-based optimizer?<\/strong><\/p>\nA rule-based optimizer uses a set of preset rules and points to determine the best approach to execute a query. The rules assign a \u201ccost\u201d to each SQL operation; the costs are then added to yield the cost of the execution plan.\n\nA cost-based optimizer uses sophisticated algorithms based on the statistics about the objects being accessed to determine the best approach to execute a query. In this case, the optimizer process adds up the processing cost, the I\/O costs and the resource costs (RAM and temporary space) to come up with the total cost of a given execution plan.\n<p class=\"page-title\"><strong>What are optimizer hints and how are they used?<\/strong><\/p>\nHints are special instructions for the optimizer that are embedded inside the SQL command text. Although the optimizer generally performs very well under most circumstances, there are some circumstances in which the optimizer may not choose the best execution plan. Remember, the optimizer makes decisions based on the existing statistics. If the statistics are old, the optimizer may not do a good job in selecting the best execution plan. Even with the current statistics, the optimizer choice may not be the most efficient one. There are some occasions when the end-user would like to change the optimizer mode for the current SQL statement. In order to accomplish this task, you have to use hints.\n<p class=\"page-title\"><strong>What are some general guidelines for creating and using indexes?<\/strong><\/p>\n<i>Create indexes for each single attribute used in a WHERE, HAVING, ORDER BY, or GROUP BY clause<\/i>. If you create indexes in all single attributes\u00a0<i>used in search conditions<\/i>, the DBMS will access the table using an index scan, instead of a full table scan. For example, if you have an index for P_PRICE, the condition P_PRICE &gt; 10.00 can be solved by accessing the index, instead of sequentially scanning all table rows and evaluating P_PRICE for each row. Indexes are also used in join expressions, such as in CUSTOMER.CUS_CODE = INVOICE.CUS_CODE.\n\n<i>Do not use indexes in small tables or tables with low sparsity<\/i>. Remember, small tables and low sparsity tables are not the same thing. A search condition in a table with low sparsity may return a high percentage of table rows anyway, making the index operation too costly and making the full table scan a viable option. Using the same logic, do not create indexes for tables with few rows and few attributes\u2014<i>unless you must ensure the existence of unique values in a column<\/i>.\n\n<i>Declare primary and foreign keys so the optimizer can use the indexes in join operations<\/i>. All natural joins and old-style joins will benefit if you declare primary keys and foreign keys because the optimizer will use the available indexes at join time. (The declaration of a PK or FK will automatically create an index for the declared column. Also, for the same reason, it is better to write joins using the SQL JOIN syntax. (See Chapter 8, \u201cAdvanced SQL.\u201d)\n\n<i>Declare indexes in join columns other than PK\/FK<\/i>. If you do join operations on columns other than the primary and foreign key, you may be better off declaring indexes in such columns.\n<p class=\"page-title\"><strong>What factors should you keep in mind if you intend to write conditional expressions in SQL code?<\/strong><\/p>\n<i>Use simple columns or literals as operands in a conditional expression\u2014avoid the use of conditional expressions with functions whenever possible<\/i>. Comparing the contents of a single column to a literal is faster than comparing to expressions.\n\n<i>Numeric field comparisons are faster than character, date, and NULL comparisons<\/i>. In search conditions, comparing a numeric attribute to a numeric literal is faster than comparing a character attribute to a character literal. In general, numeric comparisons (integer, decimal) are handled faster by the CPU than character and date comparisons. Because indexes do not store references to null values, NULL conditions involve additional processing and therefore tend to be the slowest of all conditional operands.\n\n<i>Equality comparisons are faster than inequality comparisons<\/i>. As a general rule, equality comparisons are processed faster than inequality comparisons. For example, P_PRICE = 10.00 is processed faster because the DBMS can do a direct search using the index in the column. If there are no exact matches, the condition is evaluated as false. However, if you use an inequality symbol (&gt;, &gt;=, &lt;, &lt;=) the DBMS must perform additional processing to complete the request. This is because there would almost always be more \u201cgreater than\u201d or \u201cless than\u201d values and perhaps only a few exactly \u201cequal\u201d values in the index. The slowest (with the exception of NULL) of all comparison operators is LIKE with wildcard symbols, such as in V_CONTACT LIKE \u201c%glo%\u201d. Also, using the \u201cnot equal\u201d symbol (&lt;&gt;) yields slower searches, especially if the sparsity of the data is high; that is, if there are many more different values than there are equal values.\n\n<i>Whenever possible, transform conditional expressions to use literals<\/i>. For example, if your condition is P_PRICE -10 = 7, change it to read P_PRICE = 17. Also, if you have a composite condition such as:\n\nP_QOH &lt; P_MIN AND P_MIN = P_REORDER AND P_QOH = 10\n\nchange it to read:\n\nP_QOH = 10 AND P_MIN = P_REORDER AND P_MIN &gt; 10\n\n<i>When using multiple conditional expressions, write the equality conditions first<\/i>. (Note that we did this in the previous example.) Remember, equality conditions are faster to process than inequality conditions. Although most RDBMSs will automatically do this for you, paying attention to this detail lightens the load for the query optimizer. (The optimizer won\u2019t have to do what you have already done.)\n\n<i>If you use multiple AND conditions, write the condition most likely to be false first<\/i>. If you use this technique, the DBMS will stop evaluating the rest of the conditions as soon as it finds a conditional expression that is evaluated to be false. Remember, for multiple AND conditions to be found true, all conditions must be evaluated as true. If one of the conditions evaluates to false, everything else is evaluated as false. Therefore, if you use this technique, the DBMS won\u2019t waste time unnecessarily evaluating additional conditions. Naturally, the use of this technique implies an implicit knowledge of the sparsity of the data set.\n\n<i>Whenever possible, try to avoid the use of the NOT logical operator<\/i>. It is best to transform a SQL expression containing a NOT logical operator into an equivalent expression. For example:\n\nNOT (P_PRICE &gt; 10.00) can be written as P_PRICE &lt;= 10.00.\n\nAlso, NOT (EMP_SEX = 'M') can be written as EMP_SEX = 'F'.\n<p class=\"page-title\"><strong>What recommendations would you make for managing the data files in a DBMS with many tables and indexes?<\/strong><\/p>\nFirst, create independent data files for the system, indexes and user data table spaces. Put the data files on separate disks or RAID volumes. This ensures that index operations will not conflict with end-user data or data dictionary table access operations.\n\nSecond, put high-usage end-user tables in their own table spaces. By doing this, the database minimizes conflicts with other tables and maximizes storage utilization.\n\nThird, evaluate the creation of indexes based on the access patterns. Identify common search criteria and isolate the most frequently used columns in search conditions. Create indexes on high usage columns with high sparsity.\n\nFourth, evaluate the usage of aggregate queries in your database. Identify columns used in aggregate functions and determine if the creation of indexes on such columns will improve response time.\n\nFinally, identify columns used in ORDER BY statements and make sure there are indexes on such columns.\n<p class=\"page-title\"><strong>What does RAID stand for, and what are some commonly used RAID levels?<\/strong><\/p>\nRAID is the acronym for\u00a0<b>R<\/b>edundant\u00a0<b>A<\/b>rray of\u00a0<b>I<\/b>ndependent\u00a0<b>D<\/b>isks. RAID is used to provide balance between performance and fault tolerance. RAID systems use multiple disks to create virtual disks (storage volumes) formed by several individual disks. RAID systems provide performance improvement and fault tolerance. Table 11.7 in the text shows the commonly used RAID levels. (We have reproduced the table for your convenience.)\n\n<strong>Raid Level 0<\/strong>:\u00a0The data blocks are spread over separate drives. Also known as striped array. Provides increased performance but no fault tolerance. Fault tolerance means that in case of failure, data could be reconstructed and retrieved. Requires a minimum of two drives.\n\n<strong>Raid Level 1:<\/strong>\u00a0The same data blocks are written (duplicated) to separate drives. Also referred to as mirroring or duplexing. Provides increased read performance and fault tolerance via data redundancy. Requires a minimum of two drives.\n\n<strong>Raid Level 3:<\/strong>\u00a0The data are striped across separate drives, and parity data are computed and stored in a dedicated drive. Parity data are specially generated data that permit the reconstruction of corrupted or missing data. Provides good read performance and fault tolerance via parity data. Requires a minimum of three drives.\n\n<strong>Raid Level 5:<\/strong>\u00a0The data and the parity are striped across separate drives. Provides good read performance and fault tolerance via parity data. Requires a minimum of three drives.\n\n<hr>\n\n<h2><a id=\"chapter12\" href=\"\"><\/a>Chapter 12 Notes for Module 8<\/h2>\n<p class=\"page-title\"><strong>Describe the different types of database requests and transactions.<\/strong><\/p>\nA database transaction is formed by one or more\u00a0<i>database requests<\/i>. Each database request is the equivalent of a single SQL statement. The basic difference between a\u00a0<i>local<\/i>\u00a0transaction and a\u00a0<i>distributed\u00a0<\/i>transaction is that the latter can update or request data from several\u00a0<i>remote sites<\/i>\u00a0on a network. In a DDBMS, a database request and a database transaction can be of two types: remote or distributed.\n\nNote: The figure references in the discussions refer to the figures found in the text. The figures are not reproduced in this manual.\n\nA\u00a0<strong><i>remote request<\/i><\/strong>\u00a0accesses data located at a single remote database processor (or DP site). In other words, an SQL statement (or request) can reference data at only one remote DP site. Use Figure 12.9 to illustrate the remote request.\n\nA\u00a0<strong><i>remote transaction<\/i><\/strong>, composed of several requests, accesses data at only a single remote DP site.\u00a0 Use Figure 12.10 to illustrate the remote transaction.\n\nAs you discuss Figure 12.10, note that both tables are located at a remote DP (site B) and that the complete transaction can reference only one remote DP. Each SQL statement (or request) can reference only one (the same) remote DP at a time; the entire transaction can reference only one remote DP; and it is executed at only one remote DP.\n\nA\u00a0<strong><i>distributed transaction<\/i><\/strong>\u00a0allows a transaction to reference several different local or remote DP sites.\u00a0 Although each single request can reference only one local or remote DP site, the complete transaction can reference multiple DP sites because each request can reference a different site. Use Figure 12.11 to illustrate the distributed transaction.\n\nA\u00a0<strong><i>distributed request<\/i><\/strong>\u00a0lets us reference data from several different DP sites. Since each request can access data from more than one DP site, a transaction can access several DP sites. The ability to execute a distributed request requires\u00a0<i>fully distributed database processing<\/i>\u00a0because we must be able to:\n<ol>\n \t<li>Partition a database table into several fragments.<\/li>\n \t<li>Reference one or more of those fragments with only one request. In other words, we must have fragmentation transparency<\/li>\n<\/ol>\nThe location and partition of the data should be transparent to the end user. Use Figure 12.12 to illustrate the distributed request.\n\nAs you discuss Figure 12.12, note that the transaction uses a single SELECT statement to reference two tables, CUSTOMER and INVOICE. The two tables are located at two different remote DP sites, B and C.\n\nThe distributed request feature also allows a single request to reference a physically partitioned table. For example, suppose that a CUSTOMER table is divided into two fragments C1 and C2, located at sites B and C respectively. The end user wants to obtain a list of all customers whose balance exceeds $250.00.\u00a0 Use Figure 12.13 to illustrate this distributed request.\n\nNote that full fragmentation support is provided only by a DDBMS that supports distributed requests.\n<p class=\"page-title\"><strong>What is the objective of the query optimization functions?<\/strong><\/p>\nThe objective of query optimization functions is to minimize the total costs associated with the execution of a database request. The costs associated with a request are a function of:\n<ul>\n \t<li>the access time (I\/O) cost involved in accessing the physical data stored on disk<\/li>\n \t<li>the communication cost associated with the transmission of data among nodes in distributed database systems<\/li>\n \t<li>the CPU time cost.<\/li>\n<\/ul>\nIt is difficult to separate communication and processing costs. Query\u2011optimization algorithms use different parameters, and the algorithms assign different weight to each parameter. For example, some algorithms minimize total time, others minimize the communication time, and still others do not factor in the CPU time, considering it insignificant relative to the other costs. Query optimization must provide\u00a0<i>distribution and replica transparency<\/i>\u00a0in distributed database systems.\n<p class=\"page-title\"><strong>To which transparency feature are the query optimization functions related?<\/strong><\/p>\nQuery\u2011optimization functions are associated with the performance transparency features of a DDBMS. In a DDBMS the query\u2011optimization routines are more complicated because the DDBMS must decide\u00a0<i>where and which fragment<\/i>\u00a0of the database to access. Data fragments are stored at several sites, and the data fragments are replicated at several sites.\n<p class=\"page-title\"><strong>What issues should be considered when resolving data requests in a distributed environment?<\/strong><\/p>\nA data request could be either a read or a write request. However, most requests tend to be read requests. In both cases, resolving data requests in a distributed data environment most consider the following issues:\n<ul>\n \t<li>Data distribution.<\/li>\n \t<li>Data replication.<\/li>\n \t<li>Network and node availability.<\/li>\n<\/ul>\nA more detailed discussion of these factors can be found in section 12.10.\n<p class=\"page-title\"><strong>What are the two basic styles of data replication?<\/strong><\/p>\nThere are basically two styles of replication:\n<ul>\n \t<li>Push replication. In this case, the originating DP node sends the changes to the replica nodes to ensure that all data are mutually consistent.<\/li>\n \t<li>Pull replication. The originating DP node notifies the replica nodes so they can pull the updates one their own time.<\/li>\n<\/ul>\nSee section 12.11.2 for more information.\n<p class=\"page-title\"><strong>What trade-offs are involved in building highly distributed data environments?<\/strong><\/p>\nIn the year 2000, Dr. Eric Brewer stated in a presentation that: \u201cin any highly distributed data system there are three common desirable properties: consistency, availability and partition tolerance. However, it is impossible for a system to provide all three properties at the same time.\u201d Therefore, the system designers have to balance the trade-offs of these properties in order to provide a workable system. This is what is known as the CAP theorem. For more information on this, see section 12.12.\n<p class=\"page-title\"><strong>How does a BASE system differ from a traditional distributed database system?<\/strong><\/p>\nA traditional database system enforces the ACID properties as to ensure that all database transactions yield a database in a consistent state. In a centralized database system, all data resides in a centralized node. However, in a distributed database system data are located in multiple geographically disperse sites connected via a network. In such cases, network latency and network partitioning impose a new level of complexity. In most highly distributed systems, designers tend to emphasize availability over data consistency and partition tolerance. This trade-off has given way to a new type of database systems in which data are basically available, soft state and eventually consistent (BASE).\n\nFor more information about BASE systems see section12.12.\n<p class=\"page-title\"><strong>Briefly describe the process of writing a file in a Hadoop Distributed File System (HDFS).<\/strong><\/p>\nA client node wishing to write a file contacts the name node.\u00a0 The name node will add the new file to the metadata, and determine which data nodes the first block of the file will be stored on.\u00a0 This data is sent back to the client node.\u00a0 The client node then contacts the first data node provided by the name node and begins writing the file to that data node.\u00a0 The client node also sends the data node the list of data nodes to which this block should be replicated.\u00a0 As the first data nodes receives the block of data, it contacts the next data node on the list and begins transmitting the block to this data node, which in turn repeats the process with the next data node on the list to replicate the block.\u00a0 When a block of the file is finished writing to the first data node, the client node contacts the name node to repeat this process for each block in the file.\n<p class=\"page-title\"><strong>Define the term heartbeat in an HDFS and explain its purpose.<\/strong><\/p>\nIn HDFS, each data node communicates with the name node periodically.\u00a0 One type of communication between the data nodes and the name node is a heartbeat.\u00a0 A heartbeat is a signal sent to the name node to let it know that the data node is still active in the file system.\u00a0 This signal is sent every 3 seconds by each data node.\u00a0 If the name node fails to receive a heartbeat from a data node, then it will decide that the data node has experienced a fault, and the name node will know not to send client nodes to that data node for read or write operations.","rendered":"<h2>Chapter 11 Notes for Module 8<\/h2>\n<p class=\"page-title\"><strong>What is SQL performance tuning?<\/strong><\/p>\n<p>SQL performance tuning describes a process \u2013 on the client side \u2013 that will generate an SQL query to return the correct answer in the least amount of time, using the minimum amount of resources at the server end.<\/p>\n<p class=\"page-title\"><strong>What is database performance tuning?<\/strong><\/p>\n<p>DBMS performance tuning describes a process \u2013 on the server side \u2013 that will properly configure the DBMS environment to respond to clients\u2019 requests in the fastest way possible, while making optimum use of existing resources.<\/p>\n<p class=\"page-title\"><strong>What is the focus of most performance tuning activities, and why does that focus exist?<\/strong><\/p>\n<p>Most performance-tuning activities focus on minimizing the number of I\/O operations, because the I\/O operations are much slower than reading data from the data cache.<\/p>\n<p class=\"page-title\">What are database statistics, and why are they important?<\/p>\n<p>The term\u00a0<i>database statistics<\/i>\u00a0refers to a number of measurements gathered by the DBMS to describe a snapshot of the database objects\u2019 characteristics. The DBMS gathers statistics about objects such as tables, indexes, and available resources &#8212; such as number of processors used, processor speed, temporary space available, and so on. Such statistics are used to make critical decisions about improving the query processing efficiency.<\/p>\n<p class=\"page-title\"><strong>How are database statistics obtained?<\/strong><\/p>\n<p>Database statistics can be gathered manually by the DBA or automatically by the DBMS. For example, many DBMS vendors support the SQL\u2019s ANALYZE command to gather statistics. In addition, many vendors have their own routines to gather statistics. For example, IBM\u2019s DB2 uses the RUNSTATS procedure, while Microsoft\u2019s SQL Server uses the UPDATE STATISTICS procedure and provides the Auto-Update and Auto-Create Statistics options in its initialization parameters.<\/p>\n<p class=\"page-title\"><strong>What database statistics measurements are typical of tables, indexes, and resources?<\/strong><\/p>\n<p>For tables, typical measurements include the number of rows, the number of disk blocks used, row length, the number of columns in each row, the number of distinct values in each column, the maximum value in each column, the minimum value in each column, what columns have indexes, and so on.<\/p>\n<p>For indexes, typical measurements include the number and name of columns in the index key, the number of key values in the index, the number of distinct key values in the index key, histogram of key values in an index, etc.<\/p>\n<p>For resources, typical measurements include the logical and physical disk block size, the location and size of data files, the number of extends per data file, and so on.<\/p>\n<p class=\"page-title\"><strong>How is the processing of SQL DDL statements (such as CREATE TABLE) different from the processing required by DML statements?<\/strong><\/p>\n<p>A DDL statement actually updates the data dictionary tables or system catalog, while a DML statement (SELECT, INSERT, UPDATE and DELETE) mostly manipulates end user data.<\/p>\n<p class=\"page-title\"><strong>What are the three phases that the DBMS processes queries in and what is accomplished in each phase?<\/strong><\/p>\n<p>The three phases are:<\/p>\n<ol>\n<li><i>Parsing<\/i>. The DBMS parses the SQL query and chooses the most efficient access\/execution plan.<\/li>\n<li><i>Execution<\/i>. The DBMS executes the SQL query using the chosen execution plan.<\/li>\n<li><i>Fetching<\/i>. The DBMS fetches the data and sends the result set back to the client.<\/li>\n<\/ol>\n<p>Parsing involves breaking the query into smaller units and transforming the original SQL query into a slightly different version of the original SQL code &#8212; but one that is \u201cfully equivalent\u201d and more efficient.\u00a0<i>Fully equivalent<\/i>\u00a0means that the optimized query results are always the same as the original query. More efficient means that the optimized query will, almost always, execute faster than the original query. (Note that we say\u00a0<i>almost<\/i>\u00a0always because many factors affect the performance of a database. These factors include the network, the client\u2019s computer resources, and even other queries running concurrently in the same database.)<\/p>\n<p>After the parsing and execution phases are completed, all rows that match the specified condition(s) have been retrieved, sorted, grouped, and\/or \u2013 if required \u2013 aggregated. During the fetching phase, the rows of the resulting query result set are returned to the client. During this phase, the DBMS may use temporary table space to store temporary data.<\/p>\n<p class=\"page-title\"><strong>If indexes are so important, why not index every column in every table?<\/strong><\/p>\n<p>Indexing every column in every table will tax the DBMS too much in terms of index-maintenance processing, especially if the table has many attributes, many rows, and\/or requires many inserts, updates, and\/or deletes.<\/p>\n<p>One measure to determine the need for an index is the data\u00a0<i>sparsity<\/i>\u00a0of the column you want to index. Data sparsity refers to the number of different values a column could possibly have. For example, a STU_SEX column in a STUDENT table can have only two possible values, \u201cM\u201d or \u201cF\u201d; therefore this column is said to have low sparsity. In contrast, the STU_DOB column that stores the student date of birth can have many different date values; therefore, this column is said to have high sparsity. Knowing the sparsity helps you decide whether or not the use of an index is appropriate. For example, when you perform a search in a column with low sparsity, you are very likely to read a high percentage of the table rows anyway; therefore index processing may be unnecessary work.<\/p>\n<p class=\"page-title\"><strong>What is the difference between a rule-based optimizer and a cost-based optimizer?<\/strong><\/p>\n<p>A rule-based optimizer uses a set of preset rules and points to determine the best approach to execute a query. The rules assign a \u201ccost\u201d to each SQL operation; the costs are then added to yield the cost of the execution plan.<\/p>\n<p>A cost-based optimizer uses sophisticated algorithms based on the statistics about the objects being accessed to determine the best approach to execute a query. In this case, the optimizer process adds up the processing cost, the I\/O costs and the resource costs (RAM and temporary space) to come up with the total cost of a given execution plan.<\/p>\n<p class=\"page-title\"><strong>What are optimizer hints and how are they used?<\/strong><\/p>\n<p>Hints are special instructions for the optimizer that are embedded inside the SQL command text. Although the optimizer generally performs very well under most circumstances, there are some circumstances in which the optimizer may not choose the best execution plan. Remember, the optimizer makes decisions based on the existing statistics. If the statistics are old, the optimizer may not do a good job in selecting the best execution plan. Even with the current statistics, the optimizer choice may not be the most efficient one. There are some occasions when the end-user would like to change the optimizer mode for the current SQL statement. In order to accomplish this task, you have to use hints.<\/p>\n<p class=\"page-title\"><strong>What are some general guidelines for creating and using indexes?<\/strong><\/p>\n<p><i>Create indexes for each single attribute used in a WHERE, HAVING, ORDER BY, or GROUP BY clause<\/i>. If you create indexes in all single attributes\u00a0<i>used in search conditions<\/i>, the DBMS will access the table using an index scan, instead of a full table scan. For example, if you have an index for P_PRICE, the condition P_PRICE &gt; 10.00 can be solved by accessing the index, instead of sequentially scanning all table rows and evaluating P_PRICE for each row. Indexes are also used in join expressions, such as in CUSTOMER.CUS_CODE = INVOICE.CUS_CODE.<\/p>\n<p><i>Do not use indexes in small tables or tables with low sparsity<\/i>. Remember, small tables and low sparsity tables are not the same thing. A search condition in a table with low sparsity may return a high percentage of table rows anyway, making the index operation too costly and making the full table scan a viable option. Using the same logic, do not create indexes for tables with few rows and few attributes\u2014<i>unless you must ensure the existence of unique values in a column<\/i>.<\/p>\n<p><i>Declare primary and foreign keys so the optimizer can use the indexes in join operations<\/i>. All natural joins and old-style joins will benefit if you declare primary keys and foreign keys because the optimizer will use the available indexes at join time. (The declaration of a PK or FK will automatically create an index for the declared column. Also, for the same reason, it is better to write joins using the SQL JOIN syntax. (See Chapter 8, \u201cAdvanced SQL.\u201d)<\/p>\n<p><i>Declare indexes in join columns other than PK\/FK<\/i>. If you do join operations on columns other than the primary and foreign key, you may be better off declaring indexes in such columns.<\/p>\n<p class=\"page-title\"><strong>What factors should you keep in mind if you intend to write conditional expressions in SQL code?<\/strong><\/p>\n<p><i>Use simple columns or literals as operands in a conditional expression\u2014avoid the use of conditional expressions with functions whenever possible<\/i>. Comparing the contents of a single column to a literal is faster than comparing to expressions.<\/p>\n<p><i>Numeric field comparisons are faster than character, date, and NULL comparisons<\/i>. In search conditions, comparing a numeric attribute to a numeric literal is faster than comparing a character attribute to a character literal. In general, numeric comparisons (integer, decimal) are handled faster by the CPU than character and date comparisons. Because indexes do not store references to null values, NULL conditions involve additional processing and therefore tend to be the slowest of all conditional operands.<\/p>\n<p><i>Equality comparisons are faster than inequality comparisons<\/i>. As a general rule, equality comparisons are processed faster than inequality comparisons. For example, P_PRICE = 10.00 is processed faster because the DBMS can do a direct search using the index in the column. If there are no exact matches, the condition is evaluated as false. However, if you use an inequality symbol (&gt;, &gt;=, &lt;, &lt;=) the DBMS must perform additional processing to complete the request. This is because there would almost always be more \u201cgreater than\u201d or \u201cless than\u201d values and perhaps only a few exactly \u201cequal\u201d values in the index. The slowest (with the exception of NULL) of all comparison operators is LIKE with wildcard symbols, such as in V_CONTACT LIKE \u201c%glo%\u201d. Also, using the \u201cnot equal\u201d symbol (&lt;&gt;) yields slower searches, especially if the sparsity of the data is high; that is, if there are many more different values than there are equal values.<\/p>\n<p><i>Whenever possible, transform conditional expressions to use literals<\/i>. For example, if your condition is P_PRICE -10 = 7, change it to read P_PRICE = 17. Also, if you have a composite condition such as:<\/p>\n<p>P_QOH &lt; P_MIN AND P_MIN = P_REORDER AND P_QOH = 10<\/p>\n<p>change it to read:<\/p>\n<p>P_QOH = 10 AND P_MIN = P_REORDER AND P_MIN &gt; 10<\/p>\n<p><i>When using multiple conditional expressions, write the equality conditions first<\/i>. (Note that we did this in the previous example.) Remember, equality conditions are faster to process than inequality conditions. Although most RDBMSs will automatically do this for you, paying attention to this detail lightens the load for the query optimizer. (The optimizer won\u2019t have to do what you have already done.)<\/p>\n<p><i>If you use multiple AND conditions, write the condition most likely to be false first<\/i>. If you use this technique, the DBMS will stop evaluating the rest of the conditions as soon as it finds a conditional expression that is evaluated to be false. Remember, for multiple AND conditions to be found true, all conditions must be evaluated as true. If one of the conditions evaluates to false, everything else is evaluated as false. Therefore, if you use this technique, the DBMS won\u2019t waste time unnecessarily evaluating additional conditions. Naturally, the use of this technique implies an implicit knowledge of the sparsity of the data set.<\/p>\n<p><i>Whenever possible, try to avoid the use of the NOT logical operator<\/i>. It is best to transform a SQL expression containing a NOT logical operator into an equivalent expression. For example:<\/p>\n<p>NOT (P_PRICE &gt; 10.00) can be written as P_PRICE &lt;= 10.00.<\/p>\n<p>Also, NOT (EMP_SEX = &#8216;M&#8217;) can be written as EMP_SEX = &#8216;F&#8217;.<\/p>\n<p class=\"page-title\"><strong>What recommendations would you make for managing the data files in a DBMS with many tables and indexes?<\/strong><\/p>\n<p>First, create independent data files for the system, indexes and user data table spaces. Put the data files on separate disks or RAID volumes. This ensures that index operations will not conflict with end-user data or data dictionary table access operations.<\/p>\n<p>Second, put high-usage end-user tables in their own table spaces. By doing this, the database minimizes conflicts with other tables and maximizes storage utilization.<\/p>\n<p>Third, evaluate the creation of indexes based on the access patterns. Identify common search criteria and isolate the most frequently used columns in search conditions. Create indexes on high usage columns with high sparsity.<\/p>\n<p>Fourth, evaluate the usage of aggregate queries in your database. Identify columns used in aggregate functions and determine if the creation of indexes on such columns will improve response time.<\/p>\n<p>Finally, identify columns used in ORDER BY statements and make sure there are indexes on such columns.<\/p>\n<p class=\"page-title\"><strong>What does RAID stand for, and what are some commonly used RAID levels?<\/strong><\/p>\n<p>RAID is the acronym for\u00a0<b>R<\/b>edundant\u00a0<b>A<\/b>rray of\u00a0<b>I<\/b>ndependent\u00a0<b>D<\/b>isks. RAID is used to provide balance between performance and fault tolerance. RAID systems use multiple disks to create virtual disks (storage volumes) formed by several individual disks. RAID systems provide performance improvement and fault tolerance. Table 11.7 in the text shows the commonly used RAID levels. (We have reproduced the table for your convenience.)<\/p>\n<p><strong>Raid Level 0<\/strong>:\u00a0The data blocks are spread over separate drives. Also known as striped array. Provides increased performance but no fault tolerance. Fault tolerance means that in case of failure, data could be reconstructed and retrieved. Requires a minimum of two drives.<\/p>\n<p><strong>Raid Level 1:<\/strong>\u00a0The same data blocks are written (duplicated) to separate drives. Also referred to as mirroring or duplexing. Provides increased read performance and fault tolerance via data redundancy. Requires a minimum of two drives.<\/p>\n<p><strong>Raid Level 3:<\/strong>\u00a0The data are striped across separate drives, and parity data are computed and stored in a dedicated drive. Parity data are specially generated data that permit the reconstruction of corrupted or missing data. Provides good read performance and fault tolerance via parity data. Requires a minimum of three drives.<\/p>\n<p><strong>Raid Level 5:<\/strong>\u00a0The data and the parity are striped across separate drives. Provides good read performance and fault tolerance via parity data. Requires a minimum of three drives.<\/p>\n<hr \/>\n<h2><a id=\"chapter12\" href=\"\"><\/a>Chapter 12 Notes for Module 8<\/h2>\n<p class=\"page-title\"><strong>Describe the different types of database requests and transactions.<\/strong><\/p>\n<p>A database transaction is formed by one or more\u00a0<i>database requests<\/i>. Each database request is the equivalent of a single SQL statement. The basic difference between a\u00a0<i>local<\/i>\u00a0transaction and a\u00a0<i>distributed\u00a0<\/i>transaction is that the latter can update or request data from several\u00a0<i>remote sites<\/i>\u00a0on a network. In a DDBMS, a database request and a database transaction can be of two types: remote or distributed.<\/p>\n<p>Note: The figure references in the discussions refer to the figures found in the text. The figures are not reproduced in this manual.<\/p>\n<p>A\u00a0<strong><i>remote request<\/i><\/strong>\u00a0accesses data located at a single remote database processor (or DP site). In other words, an SQL statement (or request) can reference data at only one remote DP site. Use Figure 12.9 to illustrate the remote request.<\/p>\n<p>A\u00a0<strong><i>remote transaction<\/i><\/strong>, composed of several requests, accesses data at only a single remote DP site.\u00a0 Use Figure 12.10 to illustrate the remote transaction.<\/p>\n<p>As you discuss Figure 12.10, note that both tables are located at a remote DP (site B) and that the complete transaction can reference only one remote DP. Each SQL statement (or request) can reference only one (the same) remote DP at a time; the entire transaction can reference only one remote DP; and it is executed at only one remote DP.<\/p>\n<p>A\u00a0<strong><i>distributed transaction<\/i><\/strong>\u00a0allows a transaction to reference several different local or remote DP sites.\u00a0 Although each single request can reference only one local or remote DP site, the complete transaction can reference multiple DP sites because each request can reference a different site. Use Figure 12.11 to illustrate the distributed transaction.<\/p>\n<p>A\u00a0<strong><i>distributed request<\/i><\/strong>\u00a0lets us reference data from several different DP sites. Since each request can access data from more than one DP site, a transaction can access several DP sites. The ability to execute a distributed request requires\u00a0<i>fully distributed database processing<\/i>\u00a0because we must be able to:<\/p>\n<ol>\n<li>Partition a database table into several fragments.<\/li>\n<li>Reference one or more of those fragments with only one request. In other words, we must have fragmentation transparency<\/li>\n<\/ol>\n<p>The location and partition of the data should be transparent to the end user. Use Figure 12.12 to illustrate the distributed request.<\/p>\n<p>As you discuss Figure 12.12, note that the transaction uses a single SELECT statement to reference two tables, CUSTOMER and INVOICE. The two tables are located at two different remote DP sites, B and C.<\/p>\n<p>The distributed request feature also allows a single request to reference a physically partitioned table. For example, suppose that a CUSTOMER table is divided into two fragments C1 and C2, located at sites B and C respectively. The end user wants to obtain a list of all customers whose balance exceeds $250.00.\u00a0 Use Figure 12.13 to illustrate this distributed request.<\/p>\n<p>Note that full fragmentation support is provided only by a DDBMS that supports distributed requests.<\/p>\n<p class=\"page-title\"><strong>What is the objective of the query optimization functions?<\/strong><\/p>\n<p>The objective of query optimization functions is to minimize the total costs associated with the execution of a database request. The costs associated with a request are a function of:<\/p>\n<ul>\n<li>the access time (I\/O) cost involved in accessing the physical data stored on disk<\/li>\n<li>the communication cost associated with the transmission of data among nodes in distributed database systems<\/li>\n<li>the CPU time cost.<\/li>\n<\/ul>\n<p>It is difficult to separate communication and processing costs. Query\u2011optimization algorithms use different parameters, and the algorithms assign different weight to each parameter. For example, some algorithms minimize total time, others minimize the communication time, and still others do not factor in the CPU time, considering it insignificant relative to the other costs. Query optimization must provide\u00a0<i>distribution and replica transparency<\/i>\u00a0in distributed database systems.<\/p>\n<p class=\"page-title\"><strong>To which transparency feature are the query optimization functions related?<\/strong><\/p>\n<p>Query\u2011optimization functions are associated with the performance transparency features of a DDBMS. In a DDBMS the query\u2011optimization routines are more complicated because the DDBMS must decide\u00a0<i>where and which fragment<\/i>\u00a0of the database to access. Data fragments are stored at several sites, and the data fragments are replicated at several sites.<\/p>\n<p class=\"page-title\"><strong>What issues should be considered when resolving data requests in a distributed environment?<\/strong><\/p>\n<p>A data request could be either a read or a write request. However, most requests tend to be read requests. In both cases, resolving data requests in a distributed data environment most consider the following issues:<\/p>\n<ul>\n<li>Data distribution.<\/li>\n<li>Data replication.<\/li>\n<li>Network and node availability.<\/li>\n<\/ul>\n<p>A more detailed discussion of these factors can be found in section 12.10.<\/p>\n<p class=\"page-title\"><strong>What are the two basic styles of data replication?<\/strong><\/p>\n<p>There are basically two styles of replication:<\/p>\n<ul>\n<li>Push replication. In this case, the originating DP node sends the changes to the replica nodes to ensure that all data are mutually consistent.<\/li>\n<li>Pull replication. The originating DP node notifies the replica nodes so they can pull the updates one their own time.<\/li>\n<\/ul>\n<p>See section 12.11.2 for more information.<\/p>\n<p class=\"page-title\"><strong>What trade-offs are involved in building highly distributed data environments?<\/strong><\/p>\n<p>In the year 2000, Dr. Eric Brewer stated in a presentation that: \u201cin any highly distributed data system there are three common desirable properties: consistency, availability and partition tolerance. However, it is impossible for a system to provide all three properties at the same time.\u201d Therefore, the system designers have to balance the trade-offs of these properties in order to provide a workable system. This is what is known as the CAP theorem. For more information on this, see section 12.12.<\/p>\n<p class=\"page-title\"><strong>How does a BASE system differ from a traditional distributed database system?<\/strong><\/p>\n<p>A traditional database system enforces the ACID properties as to ensure that all database transactions yield a database in a consistent state. In a centralized database system, all data resides in a centralized node. However, in a distributed database system data are located in multiple geographically disperse sites connected via a network. In such cases, network latency and network partitioning impose a new level of complexity. In most highly distributed systems, designers tend to emphasize availability over data consistency and partition tolerance. This trade-off has given way to a new type of database systems in which data are basically available, soft state and eventually consistent (BASE).<\/p>\n<p>For more information about BASE systems see section12.12.<\/p>\n<p class=\"page-title\"><strong>Briefly describe the process of writing a file in a Hadoop Distributed File System (HDFS).<\/strong><\/p>\n<p>A client node wishing to write a file contacts the name node.\u00a0 The name node will add the new file to the metadata, and determine which data nodes the first block of the file will be stored on.\u00a0 This data is sent back to the client node.\u00a0 The client node then contacts the first data node provided by the name node and begins writing the file to that data node.\u00a0 The client node also sends the data node the list of data nodes to which this block should be replicated.\u00a0 As the first data nodes receives the block of data, it contacts the next data node on the list and begins transmitting the block to this data node, which in turn repeats the process with the next data node on the list to replicate the block.\u00a0 When a block of the file is finished writing to the first data node, the client node contacts the name node to repeat this process for each block in the file.<\/p>\n<p class=\"page-title\"><strong>Define the term heartbeat in an HDFS and explain its purpose.<\/strong><\/p>\n<p>In HDFS, each data node communicates with the name node periodically.\u00a0 One type of communication between the data nodes and the name node is a heartbeat.\u00a0 A heartbeat is a signal sent to the name node to let it know that the data node is still active in the file system.\u00a0 This signal is sent every 3 seconds by each data node.\u00a0 If the name node fails to receive a heartbeat from a data node, then it will decide that the data node has experienced a fault, and the name node will know not to send client nodes to that data node for read or write operations.<\/p>\n","protected":false},"author":11,"menu_order":8,"template":"","meta":{"pb_show_title":"","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[49],"contributor":[],"license":[],"class_list":["post-34","chapter","type-chapter","status-publish","hentry","chapter-type-numberless"],"part":19,"_links":{"self":[{"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/chapters\/34","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/wp\/v2\/users\/11"}],"version-history":[{"count":1,"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/chapters\/34\/revisions"}],"predecessor-version":[{"id":35,"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/chapters\/34\/revisions\/35"}],"part":[{"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/parts\/19"}],"metadata":[{"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/chapters\/34\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/wp\/v2\/media?parent=34"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/pressbooks\/v2\/chapter-type?post=34"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/wp\/v2\/contributor?post=34"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/pressbooks.usnh.edu\/databasemanagementsystems\/wp-json\/wp\/v2\/license?post=34"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}