Indexes can be great for boosting the performance of a query, but if an index is never used, it can drag down an instance’s performance. This is because unused indexes must be maintained just like any index. For example, anytime a row is inserted, updated, or deleted, indexes must be maintained. If a lot of records are updated or inserted into a table, an index may experience page splitting. During index rebuilding or reorganizing, indexes are defragmented. During DBCC CHECK DB, indexes have to be checked for their integrity. During all of these processes, all data modifications have to be logged in the transaction log. And of course, indexes take up space in the data cache and on disk. All of these resources are wasted on unused indexes, and can hurt the overall performance of an instance of SQL Server.

Now imagine for a moment that a database has lots of unused indexes, and some of the indexes have many multiple columns, and thus are very wide? I have seen databases where virtually every column in every table had an index, just because the developer thought the “more indexes, the better”. I have also seen many indexes with 4, 5 6, or even more columns that are part of the index key. In situations like this, unused indexes can be a huge hidden performance problem.

Of course, your databases or not like that? Or are they? When was the last time you evaluated the indexes in your databases to see if they were being used or not? According to a recent poll of visitors on my website, nearly 45% of responders said that they have never dropped an index on a database. This could be for two reasons. First, because they have never looked for unused indexes, or because they have looked, but didn’t find any unused indexes to drop. I am guessing that most of them have never looked. While this seems like a large number, about 55% of the responders have dropped an index in the past year, which means that most DBAs are familiar with the issue of unused indexes.

I think it is important to periodically review all of the indexes in your databases in order to identify unused indexes. With the introduction of the sys.dm_db_index_usage_stats DMV in SQL Server 2005, this has become relatively easy. You can choose to run SELECT * on the DMV to return all the results, although the raw results are hard to read. Instead, you might want to use a more useful script such as the one written by Glenn Berry, or one of the many dozens of similar scripts available for identifying unused indexes by searching the Internet.

When you are evaluating which indexes are not used or not, you must keep in mind that this DMV only tracks data on index usage since the instance was last restarted. For example, if you ran Glenn’s query just after restarting your server, then most of the indexes will have not been used. Instead, you must wait for your server to be up long enough to be representative of how indexes are used in your environment. This might mean that your server needs to be running for a week, a month, even a quarter if you have many reports that are only run at the end of a quarter. If your time frame is not representative, you may end up dropping indexes that are used, but less often. Another option to consider when collecting index statistics usage over time is to run a periodic job that collects the data over any time period you like, and store it in a database for later analysis.

And that brings up another area to discuss. Should you drop indexes that are only being used occasionally? For example, perhaps a report is only run once a month, once a quarter, or even once a year. Should you keep this indexes, or drop them? This is a question I can’t answer for you, as each situation is different. For example, if an index that is used for a once a month report takes 5 seconds with the index, and 30 seconds without the index, is the benefit of that index, saving 25 seconds once a month, worth the overhead of maintaining that index? This is only a question you can answer based on the business needs of your environment.

So if you are one of the DBAs who have dropped an index in the past year, good for you. If you have never dropped an index, then I suggest you run Glenn’s query on your databases and then decide for yourself if you need to drop some unused indexes.

Every since the first file was written to a floppy disk drive using DOS, physical file fragmentation has been a problem. Essentially, when the OS writes a file to a disk subsystem, and if contiguous clusters are not available, they are written elsewhere on disk. So when a file is stored on disk in a non-contiguous manner on a disk subsystem, the file is considered to be physically fragmented. Physical file fragmentation can contribute to an additional load on your I/O subsystem and reduce performance because the disks have to work harder to read and write data. This is because the heads on disk drive have to randomly jump around to different physical locations on the disk to find all of the data, instead of reading and writing the data sequentially, as when the data is contiguous.

Physical file fragmentation can be slight, say when a single file is divided into only a handful of locations on a subsystem. In other cases, a single physical file might be divided into thousands of fragments, resulting in heavy physical file fragmentation. The greater the physical file fragmentation, the harder disks have to work, and the greater overhead that is incurred. Ideally, for best I/O performance, files should be written as contiguously as practical.

How physical file fragmentation affects SQL Server’s performance varies. For example, if a single random read or write needs to be performed, the degree of physical file fragmentation will have little or no effect on I/O performance, as the amount of work to perform a single read or write is not significantly affected by physical file fragmentation. On the other hand, if SQL Server needs to read or write a large quantity of sequential data to or from disk, and the file is heavily fragmented, then the disks will have to randomly read or write the data, even though the data is logically sequenced, resulting in extra work for the disk drives, hurting I/O performance.

Another factor that can affect how SQL Server and the disk subsystem interact is if there is a large cache sitting between SQL Server and this subsystem, or if the disk subsystem in made up of SSD or similar drives. Both of these can significantly mask the negative effects of physical file fragmentation, although it is not eliminated.

So what does all of this mean to DBAs? Because of the complexities of measuring the negative impact of physical file fragmentation on SQL Server’s performance, I make the assumption that it is always bad and do my best to prevent it in the first place. As a DBA, I always assume the worse and plan for it.

Here’s what I like to do. Before I create any new database (MDF, NDF, or LDF files), I first use the Windows defrag.exe tool to see if there is any physical file fragmentation on the disk array where I want to create my database files. If there is, then I use defrag.exe, or a third-party tool, to defrag the physical files before creating my new database. Then, I pre-size my database files to a size that I think will suffice for the next year or so. For example, if I estimate that my MDF file will be 500 GB in size in the next 12 months or so, then that is the size I create the database now. The same with the other database files.  By pre-sizing my database files, auto growth doesn’t have to kick in, which is one of the biggest causes of disk fragmentation. In other words, by creating a large file on a disk array that is defragmented, the file is created contiguously and I can prevent physical disk fragmentation from occurring in the first place.

Now, you may have two questions. First, you may be wondering about how you go about defragging a disk array that is currently being used with production databases. The problem is, is that the defrag.exe program will only defrag closed files, not open files, such as any SQL Server MDF, NDF, or LDF files that are in production. So the dilemma is, you have identified physical file fragmentation on an existing disk array where you want to create a new database, and you want to defragment it first before creating the new database to eliminate any physical file fragmentation. Assuming that you are using defrag.exe, your only option is to shut down the production databases during a maintenance window, perform the defrag, then restart SQL Server. Yes, this is a pain, but this one time pain will not only remove any existing physical file fragmentation in your current databases, it will help prevent it from future databases. As the DBA, you have to decide if the trade-off in taking the server down for a while, and potential performance gains it can provide, is worth it.

The second question you may have is how do you know what size you should pre-size your database files when you first create them? This requires experience and an educated guess. The odds are that your guess will not be exact. You will either underestimate or overestimate the size of the database 12 months in the future. There is no way to know for sure. But if you make an attempt using your best guess, your guess will be much better than letting auto growth grow the database for you, which will most likely result in physical file fragmentation. As time passes, and you realize that your guess was wrong, you can always manually grow the database as needed to make more room. Or, if you estimated too much, I would leave the extra room (and not shrinking the file sizes), unless you are in the immediate need of additional disk space. But won’t this also lead to physical fragmentation? Yes, to a limited degree. But most likely, a slight amount of physical file fragmentation will never be noticed.

The times where I have seen physical file fragmentation become a real problem is when an initial database is created with the default settings, where the MDB is set to a 1MB auto growth and the LDF is set to the 10% auto growth. If such a database is not pre-sized, but grows to multiple gigabytes over time, physical file fragmentation can become a substantial problem, especially if there are multiple databases, all using the default setting, on the same SQL Server instance.

In other words, a little physical file fragmentation is nothing to worry about. But a large amount of physical file fragmentation, often due to poor database management, can end up hurting your SQL Server’s I/O performance more than you might expect.

So at the PASS Summit this past November, when Microsoft Codename Atlanta was announced, I was intrigued, as this new tool seemed to overlap what I was trying  to accomplish in my book. Now that I have reviewed Atlanta, I don’t have anything to worry about. What Atlanta offers, and what my book will offer, are substantially different.

So what is Microsoft Codename Atlanta? Essentially, Atlanta is a cloud-based, SQL Server configuration-monitoring tool that proactively checks your SQL Server 2008 and SQL Server 2008 R2 instances for a limited number of potential configuration problems, and provides you with alerts about these problems, along with helpful information on what the alerts mean and how to correct them. The tool can also be used by Microsoft CSS to view your SQL Server’s configuration, making it easier for them to help resolve problems should you have to contact them for help with a difficult problem.

In this article, I’ll take a brief look at the beta version of Atlanta, showing you how it works, how to install it, how to use it, and show you some of the alerts it produces. Since this product is in beta, its features could change by the time you read this article, so the best way to learn about this tool is to download it and try it out yourself.

Why is it called Microsoft Codename Atlanta? Because the tool is currently in beta and because Microsoft likes to give new products codenames while they are under development. When the product is finally released, it will be given an official name. For the purposes of this article, I will refer to it simply as Atlanta.

Atlanta is supposed to launch sometime in the first half of 2011, and its licensing cost has not yet been made public.

Atlanta High-Level Overview

In brief, here’s how Atlanta works. First, you install an agent (Systems Center Operations Agent 2007 R2) on every SQL Server box. For example, if you have 10 SQL Server instances on 10 different boxes, then you will have to install the agent 10 different times. In addition, at least one server needs to be designated as a Gateway. A Gateway (another service) is used to collect data from each of the agents, and once every 24 hours, the collected data is sent to the Atlanta cloud, where data is analyzed and aggregated. Once the data is analyzed for potential configuration issues, then you can view the alerts for each of your monitored instances by accessing the Atlanta web portal via a web browser. At this point, you can view the alerts, learn about them, and then decide what action, if any, you want to take based on the alert. Each of these will be described later in this article.

Prerequisites for Installing Atlanta

Before you can install and use Atlanta, you have to ensure your monitored SQL Servers have the following software:

• Windows Server 2008 or later (32- or 64-bit)
• SQL Server 2008 or later (32- or 64-bit)
• A web browser that supports Silverlight 4.0 (which also must be installed). This can be installed on a monitored SQL Server, or on any computer with access to the Internet.

See this webpage for detailed Atlanta requirements.

Installing Atlanta

Before you can do anything with Atlanta, you must go to the Atlanta website and create an account using a Windows Live ID. If you don’t already have a Windows Live ID, then you will have to create one before creating an Atlanta account.

Once you have an Atlanta account and log in, you will see a screen like the one below.

As you can see above, installing Atlanta is a multi-step process. First, you must download what is called the Registration Certificate. It is required when you install a Gateway. Second, you must download the Atlanta Gateway and Agent Setup program, which allows you to install the Gateway, the Agent, or both. Third, you must select a box where the Gateway is to be installed. And last, you must install the agent on each SQL Server box you want monitored.

As I have been testing Atlanta, I have been running both the Gateway and the Agent on a single SQL Server box to keep things simple. Once Atlanta is released, I would suggest that you run the Gateway on a server not running a production SQL Server instance so that its overhead will not negatively affect SQL Server’s performance.

After downloading the two files, I started the setup program, which opens to a command prompt, as shown below.

As you can see, the setup program allows you to install the Agent, the Gateway, or both on a server. For my testing, I chose to install both the Gateway and Agent on the same test box.

After I selected “Both” from the above screen, the following wizard appeared.

This is just a splash screen, so I clicked on “Next.”

The first step is to install the Gateway. To do this, you must point at the Registration Certificate file you downloaded previously, which I have done above. Then I clicked “Next”.

The next screen to appear (see above) is a confirmation screen, and I clicked on “Install”.

After a brief time, the above screen appears, telling you that the installation is completed. I clicked on “Finish”.

After clicking on “Finish”, the command prompt which originally appeared now requires that you “Press any to key to continue”. When I did this, the command prompt disappeared and my simple, one server test box was configured to run Atlanta.

Atlanta offers some rudimentary configuration options by directly editing the registry. How to do this is explained in the documentation, but you would probably be wise not to make any changes, unless of course, you love to tinker and break things.

Using Atlanta

Of course, now I was excited. I wanted to go to the “Alerts” screen to see what Atlanta had discovered about my test box. Unfortunately, I was disappointed. This is because it take about 24 hours after installing Atlanta before it sends the data to the cloud, processes it, and returns you any alerts.

So the next day I logged back onto the Atlanta Website Portal and I got my first taste of the kinds of alerts that Atlanta can produce.

Above, you can see that Atlanta provided 12 different alerts. The first 6 alerts tell me that I haven’t performed a DBCC CHECKDB for 6 of the databases on this SQL Server instance. The next 5 alerts tell me that I haven’t performed a database backup on 5 of the databases on this SQL Server instance. And the last alert tells me that I need to add a missing hotfix and to also add a trace flag in order to active the hotfix.

How many alerts are included in Atlanta? I don’t know and I couldn’t find out. Hopefully, Microsoft will eventually publish this data for us.

While each of these are important pieces of information, I was a little underwhelmed at the alerts that were provided. I assume that the lack of additional alerts is because Atlanta is still in beta and that all of the alerts that will be in shipping version of the product are not all currently implemented. As a comparison, I ran the Microsoft SQL Server 2008 R2 Best Practices Analyzer (BPA) against this same server, and it found quite a few more problems with my test server than did Atlanta, as you can see below.

As you can see, the BPA, in its current form, can provide much more useful results than the existing version of Atlanta. I am assuming that when Atlanta is finally released, that its features will match, if not exceed, what the BPA offers.

Now let’s go back to the Atlanta alerts screen and take a closer look.

Above, on the alerts screen, I have highlighted the first alert. Notice that at the bottom of the screen is a brief description of the alert. It is a little hard to read in the above screen shot, so I have displayed in below in a more readable form.

As you can see, this is a straight-forward description of the alert. In addition, I am told that it has been 37 days since the DBCC CHECKDB has been run on this database. Also notice that the information you see above is from the Detail tab, and that there are two additional tabs to view. So let’s take a look at them.

The Solution tab lists a Knowledge Base (KB) URL you can click, in addition to providing a link to contact Microsoft support via email, online, or by phone. If you click on the KB link, you are directed to this webpage.

Although you can only read part of this page, notice at the bottom of the page is a reference to the BPA. In other words, the KB article you are being referred to is the same KB article that you would be referred to if you were using the BPA for checking out your instance’s configuration. The explanation on the page is very good, and I would assume that these pages will be eventually modified to reflect Atlanta content.

If you click on the link to contact Microsoft support via email, online, or by phone, you have to first login using your Live ID, and the following screen appears.

This screen allows you to select the topic of your question, and once you select a general support topic, you are then asked for a more detailed sub-topic (not shown here). Once you have selected the topic and sub-topic, the following screen appears.

As you might expect, this is where you may have to lay out some money, or use up some of your prepaid support calls. So you only want to use this option if you are willing to pay for help.

Now that we have looked at the Details and the Solution tabs, let’s look at the Context tab, which is shown below.

This is obviously an XML file that describes the Alert. The documentation does not currently describe the purpose of this tab.

Now that we have taken a brief look at what one Alert looks like, there is one more part of the Alert screen that I want to mention. Notice that the Alert below has a Status of Active, and an Occurrence of 2.

This means the Alert has not been dealt with (Status is Active) and that the Alert has fired 2 times (Occurrence = 2). Once an Alert starts, and if the cause of the Alert is not fixed, the Alert’s Occurrence will increment until the problem is fixed. If an Alert has been corrected, or if you choose to ignore the Alert, you can choose to select the Alert and the click on Close. This will remove the Alert from the screen, until, of course, should the Alert reoccur. The Activate option allows you to reactive a previously  closed Alert, although this option doesn’t appear be  working with the beta version I was using for this article.

Exploring More of the Atlanta User Interface

As you may have noticed, that there a series of tabs to the right of the screen, as show below. Up until this time, we have only  looked at the Alerts tab. So let’s check out the other tabs and see what they do.

The second tab is called the Configuration tab and displays current configuration settings that have been collected by Atlanta about your SQL Server instance.

When you open up any of the lists, you will see all the data that Atlanta collects about your SQL Server instance.

The third tab is called the Configuration Change History tab, which shows you not only the Update Value (or current value) of the various configuration settings collected by Atlanta, but the previous value as well. Because this is a new installation, the Previous Value column doesn’t have any useful information.

On the other hand, once an instance is up and running for some time, and configurations changes tracked by Atlanta are made, then Atlanta will keep a history of the changes, providing you an audit trail of sorts. Of course, this is not a complete audit trail of all the changes on your instance, plus it doesn’t show who made the change. But at least does let you know if  some key configuration settings have changed over time.

The fourth tab is the Servers tab.

This screen shows all of the Gateways and any instances of SQL Server that are running the Agent. This screen also allows you to add additional servers for monitoring by Atlanta, and it shows you the last time that configuration information was uploaded from the Agent to the Gateway, or from the Gateway to Atlanta for analysis.

The fifth and final tab is the Account tab, where you manage your Atlanta account.

The Account screen is self-explanatory, and is where you can delete the account once you are done experimenting with it.

As you can see, Atlanta is very straight-forward and easy to use. It’s design is simple and to the point.

Atlanta Pros & Cons

As I was testing Atlanta, one question that I kept asking myself was whether or not it is a product that I would want to use in a production environment, once it was released in its final form. Well, based on the beta, it is still too early for me to decide. Here are some pros and cons about  the tool that I think you will want to consider if you decide to evaluate Atlanta.

Pros

• Easy to install.
• Simple to use.
• The explanations of the Alerts seemed to be useful and relatively complete.
• Provides a single location to monitor your SQL Server instances.
• It’s web-based, so you can access it anywhere.
• It is integrated with Microsoft Support, which should make it easier for you to work with Microsoft Support when a major problem arises. This is because Microsoft Support can use the information collected by Atlanta to help troubleshoot your problem.

Cons

• A service has to be installed on each instance to be monitored. This not only means that it must be physically installed and updated, it takes up valuable SQL Server resources, especially memory.
• The data is collects is not real-time, so you can’t depend on it for catching problems are are happening now.
• Only a limited set of configuration information is monitored, and the number of alerts is not great. This will probably improve in the final version, but it still has a limited ability to collect important SQL Server information that should be monitored, such as performance data, or telling you if a job failed.
• Only supports Windows 2008 and SQL Server 2008 and higher, which limits its usefulness in most SQL Server shops that run many different versions of SQL Server.
• I seemed to get more alerts (false positives) than I would have expected to see. I would  like to see the ability turn off a alert, or configure an alert for my specific needs, which can’t be done now.
• Although not a lot of data about your SQL Server instances is sent over the Internet, some security-conscious shops may not feel comfortable sending unsecured data about their SQL Server over the Internet.
• If Internet connectivity is lost, then Atlanta is unavailable.

Summary

One big unknown now is how much Atlanta will cost. If the cost is very low, then it might be a worthwhile product. On the other hand, if it is expensive, then I don’t see much of a future for the product, especially given its limited functionality. If I were Microsoft, I would give this service away for free, just as they give the SQL Server 2008 R2 Best Practices Analyzer away for free. This way, DBAs will have a tool available that Microsoft Support can use to better help them when difficult problems arise.

Additional Resources

Microsoft Codename Atlanta Official Website

Atlanta Team Blog

Atlanta Microsoft System Center Website

Atlanta Connect Website

Whenever I speak about database maintenance, I always recommend that DBAs backup up their production database transaction logs at least once an hour. As is with most things DBA-related, there is no exact answer that will meet every SQL Server environment’s needs. The interval between transaction log backups depend on many factors, including, but not limited, to how much data the organization is willing to loose, how active the database is, and so on.

When I set up database maintenance plans myself, I generally backup at 15 minute intervals, although sometimes I deviate from this, depending on a variety of different factors.

In my most recent poll, I asked DBAs how often they backup their transaction logs, and the results are shown at the left.

It is interesting to see the various choices made by the various DBAs who completed the poll.  Based on my recommendation of backing up transaction logs at least once an hour, (if you add up the results), nearly 77% follow this recommendation, which I think is great.

A few other DBAs choose different transaction log periods. Just over 2% of the DBAs said they never back up their transaction log. I hope this is because their databases use the Simple Recovery Model, and not because they don’t know what they are doing.

A surprising 6% or so said that they don’t even know what a transaction log backup is. Hopefully, after seeing this post, they will do some research and find out why transaction log backups are so important.

I recently ran a poll, asking DBAs how often they ran DBCC CHECKDB against their databases. Compared to some of the polls I have run recently, I was somewhat pleasantly surprised by the results.

My first pleasant surprise was that at least 90% of those polled knew what DBCC CHECKDB was. This is a good sign that most DBAs have at least a basic understanding of database maintenance best practices.

Of the 90% of the DBAs who run DBCC CHECKDB against their databases, just about 27% of them do it on a daily basis. These must be very diligent DBAs who take their roles of “protectors of the organization’s data” very seriously, and I commend them.

On the other hand, the largest number of poll takers, about 40% of them, indicated that they only perform this operation weekly. These DBAs are making the assumption, that under the worst case scenario, that is acceptable to loose up to a week’s worth of their data. This is because database corruption could occur sometime between the weekly DBCC CHECKDB checks, and not be detected until the next weekly run. During this time, the corruption, if not discovered, could become part of their database backups. This means that even if the database is backed up daily, and transaction made logs regularly, that if the backups contain the corruption, and the corruption is not fixable, then they potentially might have to restore their last good backup, which in the worst case, could be one week ago.

About 10% of the DBAs responded that they run DBCC CHECKDB monthly, meaning that these DBAs are willing to risk losing, under the work case scenario, a months worth of their data.

And while the overall results of this poll were positive, there were still another roughly 19% of the poll takers who said they never run DBCC CHECKDB. Since these DBAs did not respond that they didn’t know what it was, I am assuming that many of these DBAs don’t fully understand the implications of not running DBCC CHECKDB, or that they don’t care about protecting their organization’s data (which I hope is not the case).

Of course, running DBCC CHECKDB is a resource-intensive task, and it is not always possible to run it daily on a production server, especially on very busy and large databases, and the DBA has to weigh the risk of database corruption versus the resource cost of running the command. On the other hand, DBAs can always restore their daily backups onto a non-production server and run DBCC CHECKDB on the restored backup to offload the workload. In fact, this is my personal recommendation, and in most cases is easy to implement. By offloading DBCC CHECKDB to a non-production server, you can not only perform this task daily, you also remove any potential negative performance impact it might have when running on a production server.

Database maintenance includes a defined set of proactive tasks that a DBA needs to perform on a periodic basis to help ensure that their databases perform optimally and maintain high availability. In many cases, the individual maintenance steps taken might not, by themselves, seem all that important, but the accumulative effect of performing  database maintenance tasks can be huge.

The goal of this checklist is to help you document how you are currently maintaining your databases, and at the same time to help you establish database maintenance best practices. In most cases, you will answer most of the checklist items on a database-by-database basis.

As with all my previous checklists, it is in a spreadsheet form that allows you to modify it to meet your specific needs.

I would like your input on my SQL Server Database Maintenance Checklist. For example, I would like your input on:

• What is missing from the list? What other aspects of SQL Server database maintenance should be included? Keep in mind that I can’t include every possible variation.
• What on the list could be removed because it is not very important? Keep in mind that the list is designed to be generic, so there will be items on the list that will not be applicable to all SQL Server environments.
• Does my wording make sense, or should I change any of the wording so that it is more understandable or more accurate? Should I change some of the categories, or move some of the items to different categories?

Please add your comments below, and as I get feedback, I will update the checklist. Thanks!

In my most recent poll, I asked DBAs if they have turned on instant file initialization on their SQL Servers. I was a little surprised by the results. Just over half of the DBAs who responded to the poll said that they don’t know what instant file initialization is, and another 13% said that they don’t use it. Only about a third said they have it turned on.

Instant file initialization seems to be one of SQL Server’s most hidden performance gems, and should be turned on for virtually every SQL Server. To learn more about instant file initialization, see my blog post: “Instant File Initialization Speeds SQL Server”.