Archive | February, 2011

The Magic of the Self-Assigned Lab (SQLSkills Immersion Training Days 2 and 3)

The duck has obtained scuba gear-- it can go to new depths on its own.

Today, more blog from SQLSkills Immersion Training on Internals and Performance in Dallas, TX. For more about the training, see my prior post.

Where We’ve Been

We’ve moved through most of Module 8. We’ve been through the land of the transaction log, locking and blocking, isolation and versioning, table design and partitioning, and many neighborhoods of index internals.

A Sample Case

I love learning about how the transaction log works. I know a bit about how the log operates, and as a DBA I keep this knowledge pretty fresh. So I was looking forward to yesterday’s session on transaction logging quite a bit.

This week, I’ve learned a lot about the internals of how the log works with crash recovery, as well as how log buffers work in general. (And I mean a lot. Pages and pages of notes.)

Looking over my notes, what makes me happy is how much I was able to note down and absorb much more than I have been in trainings previously. This is attributable to three things:

  • A lot of time devoted to the topic;
  • Plenty of room for questions (and there were lots!) with lots of rich data in the answer;
  • A good context for the information.

I’m really proud that I know enough now that I’m able to understand and note this level of detail. That’s a great feeling.

The magic of the Self Assigned Lab

When I’m in a great session or a good conversation and I learn something that works differently than I thought, or something very interesting, I made a note to myself with a “To Do”. These are basically self-assigned lab assignments: I’ve learned from blogging and presentations that I learn a ton by setting up my own scenarios and working to show that something works (and look at how it works), or the opposite.

I already have enough self-driven labs to keep me learning, and blogging about the best parts, for several months. Here are a few:

  1. ToDo: find an automation opportunity with DBCC PAGE WITH TABLERESULTS
  2. ToDo: create a scenario where you can’t get rid of a secondary filegroup without unusual operations.
  3. ToDo: Look at/experiment with transaction savepoints

There are lots more good ones, but I’m hoarding quite a few of them. (I have 41 at the time of this writing.)

That’s the magic of great training– not only do you pick up a lot and you receive pre-designed labs you can learn from, but you also find paths you hadn’t imagined to explore and create tools on your own. And you’re inspired to go there.

Comments { 7 }

From Record Structures To TempDb in 9 Hours or Less (SQLSkills Immersion Training Day 1)

This week I’m one lucky duck: I’m thrilled to be at SQLSkills Immersion Training on Internals and Performance in Dallas, TX.

I'm on the road this week, so expect cave-level drawings. This is a duck with a snorkel. No really, that's what it is.

Immersed in What?

I’m attending Part 1 of a series of intensive SQL Server trainings. This week is foundational, but is not a pre-requisite for other courses. Good news: you can choose to collect them all or do some classes individually, and you can do them in any order. You may be interested in taking these courses if you’re working toward becoming a Microsoft Certified Master in SQL Server (info), but you may also just want very in-depth training to help you do great work.

This week’s training is taught by Paul Randal (b | t | bio) and Kimberly Tripp (b | t | bio) of SQLSkills (@sqlskills).

Things you need to know about Paul and Kimberly: they are approachable, they are enthusiastic to hear your questions and talk to you about SQL Server, and they are great at helping you look at things in new and interesting ways.

What to Expect from Day 1

When you arrive at your first morning of Immersion training, you’ll get a healthy breakfast, a very friendly welcome, and then a giant dose of SQL Server.

This training started with record structures. We moved on and talked about more things, and tied it back to record structures. We moved on and talked about more things, then tied it back to storage internals. Then that happened again a few times– and I don’t mean there was any repetition at all. There wasn’t: we moved, and fast. We made it through modules 1, 2, and 3.

I really liked the material today because of the recurring themes and demos which built on each other. We covered a lot of ground, but it was tied together by a continuous thread of focusing on data internals and storage– it helped make a lot of different features, and options, coherent. The training built a real context to frame the information, which really helps me retain it.

How Much Did I Learn?

Writing helps me remember things, so I bought a laptop I love to type on. Today, I was happy to have it.

You don’t necessarily need to type notes at Immersion training: you get a lot of material to take home. You get a binder with printouts of the slides which contain loads of information for your reference. You get a DVD with Virtual PCs with labs. But there’s also much more information which comes about in conversation, and in discussing diagrams on the whiteboard: making notes is my best chance to help capture memories of that which I can recall later on.

What’s my guideline for making a note? It needs to be something I didn’t know at all, something I couldn’t explain to someone with confidence, or a nuance that’s easy to forget.

Some of what I learned today was very practical. Some of it helped things I already know make more sense. Some of it I know will come in handy soon.

I took 5,014 words of notes.

You Know What’s Awesome? Being In a Class of Smart People.

Reading slides, listening and thinking, is more than enough to keep my brain busy. I don’t always have the time to form a question, although I do my best.

But other people ask great questions. Today, that happened a lot.

Thanks for doing all that work, y’all. Lots of my notes today came from conversations after people asked questions, and just about all of them were things I would not have thought to ask.

Plus, Vicky Harp (t) helped me make sense of page headers in the women’s restroom.

Now, where else is that going to happen?

Fun Stuff: Dinner with the @MidnightDBA s

Jen McCown (b | t) and Sean McCown (b | t) came by to say hi and talk SQueaL after training. A bunch of fun people went to dinner. It was rad. There was laughter. And kung fu.

I think my brain is ready for more.

Comments { 1 }

Dirty Pages and Statistics IO

Warning: The DROPCLEANBUFFERS command referenced in this post is appropriate for test systems only and impacts the entire SQL Instance. If you are new to SQL Server, please use this commands with care, and be careful to read the linked Books Online documentation. Happy testing!

You were hoping for a picture, right?

The other day I was running some test queries and looking at the number of reads, and I noticed something funny.

I was dropping clean buffers prior to running a query, but I would sometimes see that there had been no physical reads.

No physical reads? Where was the data coming from?

I was working on a small number of rows, but it still bothered me.

The output looked like this:

The Set-Up

Here’s a simple simulation of what I was doing. First, create a database and insert some values.

SET NOCOUNT ON;
SET STATISTICS IO OFF;
create database dirtyBuffers
GO
USE dirtyBuffers
GO
--Create a table and insert some values
create table dbo.testme (
	i int identity,
	j char(2000) default 'baroo'
)
GO
insert dbo.testme default values
GO 20

Then, turn on Statistics IO so we can see read information. Drop clean buffers, so data isn’t in memory. Then run a query.

SET STATISTICS IO ON;

DBCC DROPCLEANBUFFERS
GO

--Select some rows
select * from dbo.testme

It should read it from disk, right?

 

What I Forgot

I was forgetting about dirty pages. In  order to get a “cold cache”, you need to first run a CHECKPOINT command to flush dirty pages to disk, then run DBCC DROPCLEANBUFFERS to remove everything from the bufferpool. This is very well documented in Books Online.

This was easy to forget because typically I test execution of queries against a restored copy of a production database, or a dataset which isn’t changing.

What I Hadn’t Realized

I don’t think I ever specifically realized that dirty pages could be immediately re-used for query results– but it makes perfect sense. I had only thought about clean pages, which were read in for one query, to be available for re-use.

I felt a little silly when I realized this. Shouldn’t I have known this? But after thinking about it I realized: there’s little gaps like this in most everyone’s knowledge. Sometimes it takes a little bit of extra experience to notice the gap and fill it in. It happens to us all.

After rerunning the commands and including a CHECKPOINT with DBCC DROPCLEANBUFFERS, I see the expected output– a physical read.

Comments { 4 }
Corrupting databases is a lot like eating paste.

More Running with Scissors: Corrupting your Database with 823 Errors

Corrupting databases is a lot like eating paste. Delicious, delicious paste.

This weekend, a question came up on Twitter asking if there was an easy way to simulate an 823 error. It seemed like a fun task to figure out.

In a  previous post, I showed  how to corrupt your database with a Hex editor to cause 824 errors.

What’s an 823 Error?

An 823 error in SQL Server is a severe error that occurs when accessing a database file. It’s described in detail in KB2015755.

This is Useful!

This is useful for learning about corruption, and practicing responses to corruption events.
You can also use this to test configuration scripts you have for database mail, operators, and alerts, to make sure the alerts are working properly.

Disclaimer: these scripts are for test environments only/as always use these at your own risk and be careful not to eat paste.

Step 1: Create a share in Windows on your test machine

First, you need to create a shared folder in Windows.
In order for later steps to work, grant your SQL Server instance’s service account full control over the share.

Step 2: Map a the share to a network drive from the SQL Instance

The ‘Net use’ command is specific to a profile, so the easiest way to handle this is to enable XP_CMDSHELL and map the drive from SQL Server itself.

--Oh no, security nightmare!
sp_configure 'xp_cmdshell', 1
RECONFIGURE

EXEC xp_cmdshell 'net use "Y:" "\\MYMACHINE\z_testnetworkDrive" /PERSISTENT:NO'

Step 3: Create a test database with a filegroup, file, and table  on the network drive

Now we just need to create a database, and create a table on the drive.

--Create test database and add a filegroup and table on the network share
IF  db_id('TestMe') IS NOT NULL
BEGIN
	USE master;
	ALTER DATABASE TestMe SET SINGLE_USER WITH ROLLBACK IMMEDIATE
	DROP DATABASE TestMe
END

CREATE DATABASE TestMe
go

ALTER DATABASE TestMe ADD FILEGROUP FG1
ALTER DATABASE TestMe ADD FILE (NAME=f1, FILENAME='Y:\f1.ndf', SIZE=128MB)TO FILEGROUP FG1 

USE TestMe
go
CREATE TABLE t1 (
	i INT IDENTITY,
	j CHAR(200) DEFAULT 'x'
) ON FG1
go
INSERT t1 DEFAULT VALUES
GO 20

--Flush everything to disk
CHECKPOINT
GO

Step 4: Start a Loop of Reads in Another Connection

Now we want to simulate reads. Open up a new connection against your instance, and run the following commands to repeatedly read data from the t1 table. We use DROPCLEANBUFFERS to make sure we’re reading from disk each time. (We already ran a checkpoint to flush the writes to disk.)

--Run this in another connection
SET NOCOUNT ON;
GO
BEGIN
	DBCC DROPCLEANBUFFERS

	SELECT * FROM t1
END
GO 50000

Step 5: Disconnect the network drive, and voila! 823 Error.

Now, back in your first connection, disconnect the network drive with the following command:

EXEC xp_cmdshell 'net use "Y:" /DELETE /Y'

Your connection which is running reads should fail with an error like this:

The connection will automatically be terminated when the error occurs.

Don’t forget to disable XP_CMDSHELL

Like so:

sp_configure 'xp_cmdshell', 0
RECONFIGURE

That’s better.

Activity: Recover from the corruption, without bringing the network drive back online

To fully do the activity, you’ll want to add some database backups in before the “corruption” event of disconnecting the network drive. You may want to combine full, differential, and/or log backups, and change data in the table at various points between (and after) backups.

Then, practice bringing things back online. How much data will be lost in each scenario? How quickly can you bring the database online?

Another Solution- The USB Drive

You can also do this by creating the filegroup, file, and table on a USB stick, and removing the USB stick instead of unmapping the network drive.

However, I preferred this example since it’s easy to re-run from management studio itself, and no additional physical devices are required.

Comments { 2 }

What’s in a Name? Guidelines for Presentation Titles

The recent trend towards online voting to select presentations for events at SQL Server conferences got me thinking: this is a great opportunity to look at how people vote, and consider what this means about presentation titles and abstracts.

Let’s Admit It: People Don’t Read Abstracts

When there’s a large slate of sessions to choose between and the abstracts aren’t easy to see, people definitely don’t read abstracts.

This isn’t too different from physical conferences. When you’re looking at your program, you may glance at the abstracts, but most of what you check out is the title and sessions. In both cases, you’re busy doing other things and you have a lot of information to absorb– you make the best choice you can based on your limited, valuable time.

The Decision Tree

The decision tree for choosing a session starts out pretty simple.

Most people have “must-see” type choices, either based on really wanting to see the presenter, or having a real need for the session topic. We’ve all had the feeling of, “OH, I could REALLY use that session!” because it’s applicable to some specific problem in our lives.

Based on my life experience (including surviving middle school, watching elections, and running for committees), the decision tree starts like this:

  • Is the presenter someone you’ve got a nerd crush on? If so: vote
  • Otherwise, is the session something you really need? If so: vote

These are both very valid reasons to vote for sessions. People who are well known for giving great presentations definitely deserve to get votes for that track record. We have a lot of superstars, which is fantastic!

But for everything that doesn’t fall into these category, things get complicated. And that’s for most sessions.

How I’ve Been Choosing Titles

I like to choose session titles that are fun and quirky. That’s just the type of session I’d like to go see– something a little out of the ordinary.

While I care about the titles a lot, and I spend a lot of time on them, I haven’t had many other criteria other than “quirky” and “amusing.”

What I’m Changing, and Why

My criteria aren’t enough. I need to make my sessions appeal to more people other than me. After all, my presentations aren’t for me.

While my titles and abstracts should still be in my own voice, I need to add more information to them to help people understand more about what I want to give them.

Pseudo-Scientific Guidelines

From my highly unscientific observations, here are some guidelines:

  • People like Tips and Tricks.
  • People like to learn a finite number of things.
  • People like practical advice they can apply quickly.
  • People like new features.
  • People like to have difficult to explain topics explained, but they need to be broad (performance, internals, memory).
  • People like T-SQL. It’s a very common tool for everyone, we use it frequently, and we all like a good session to help us think about it differently.

When I think about how I make my own choices, this does make sense. I like all of these things, too.

This doesn’t mean my next session will be titled “The Top 10 Tips and Tricks for SQL Server Performance using T-SQL and New Features!!!111!!!1!”

I may or may not use a number in presentation titles. But I’m going to consider them.

But, when writing titles (and abstracts, although I am convinced people don’t read them), I will ask myself:

  1. Does this show I will demonstrate something practical and useful?
  2. Does this help people understand why it is applicable and/or useful and interesting to know?
  3. Is there a way I can make this more practical, while still having it  in my own voice?

And I may sometimes go with a title that I feel is a little boring, if it’s easier to understand.

Am I Selling Out?

I don’t think so. But let me know. :)

Comments { 5 }

Free Poster! Guide to SQL Server Isolation Levels

Isolation Levels in SQL Server: Free Poster

Isolation Levels in SQL Server: Free Poster

I recently presented my “(NOLOCK) for YESFUN: Games with Isolation Levels” at SQL Saturday #60 in Cleveland, Ohio. I had a great time and the organizers and attendees in Cleveland were fantastic– thanks everyone for attending and for the great questions. The talk’s resources/links are here.

I drew a poster for the presentation which is available free for everyone’s viewing and/or printing pleasure. It’s designed to print nicely on legal, but the PNG can also print on larger paper.

Download the poster at http://BrentOzar.com/go/Posters

If you’d like more information on Isolation Levels in SQL Server, check out http://BrentOzar.com/go/Isolation

Comments { 18 }

Be Prepared: Collecting Data from sp_WhoIsActive in a Table

I am a fangirl of Adam Machanic‘s sp_whoisactive stored procedure.

Be prepared. When things go wrong, you want to focus on the problem itself. Make a plan for how you

One of the great things about sp_whoisactive is that it’s very quick to generate a table to collect the data you need, then run the procedure in a loop with a wait statement to regularly collect the data you want over an interval. This usage is documented (see Adam’s post here), but I’ve chatted with several people who either didn’t know you could do this, or didn’t have a script saved.

What’s sp_whoisactive?

Sp_whoisactive is a procedure written by Adam Machanic. It can be found here: sp_whoisactive downloads

What to Do Today

Do your future self a favor: Today, set yourself up the following:

  • Create the sp_whoisactive stored procedure in a user database you use only for administration
  • Play with the script I provide below and look at the  parameters of the stored procedure
  • Save a script which creates a table for your preferred version of sp_whoisactive, then runs in a loop to store the data.

Why Do This?

Even if you’re already running monitoring tools, using sp_whoisactive to sample data from the DMVs (including query plans) can be invaluable.

When things go wrong, terribly wrong, you want to have this script in your back pocket to collect data about activity on your SQL instance in the background while you’re focusing on the problem. This will help you resolve the issue as quickly as possible, by referring periodically to the data it’s collecting. Since the data is stored, you can also use it afterward to explain the situation and prevent it from happening again.

Step 1: Create Your @destination_table

Let me just note again: this is not my original work. I don’t want to take credit for it, but I do want to advocate for it!

First, create your destination table. Sp_whoIsActive lets you generate a destination table easily. Here is a basic version which includes the current date in the table name, as well as three flags to get transaction info (tlog write info and duration), query plans, and block leaders:

DECLARE @destination_table VARCHAR(4000) ;
SET @destination_table = 'WhoIsActive_' + CONVERT(VARCHAR, GETDATE(), 112) ;

DECLARE @schema VARCHAR(4000) ;
EXEC sp_WhoIsActive
@get_transaction_info = 1,
@get_plans = 1,
@find_block_leaders = 1,
@RETURN_SCHEMA = 1,
@SCHEMA = @schema OUTPUT ;

SET @schema = REPLACE(@schema, '<table_name>', @destination_table) ;

PRINT @schema
EXEC(@schema) ;

Step 2: Create Your Loop to Periodically Log Data

You need to need to make sure to specify the same parameters for sp_whoisactive in this loop, so the output matches the schema for the table you’ve created.

This does 10 runs with a 5 second wait between runs. Depending on what’s going on, I change those numbers accordingly.

DECLARE
    @destination_table VARCHAR(4000) ,
    @msg NVARCHAR(1000) ;

SET @destination_table = 'WhoIsActive_' + CONVERT(VARCHAR, GETDATE(), 112) ;

DECLARE @numberOfRuns INT ;
SET @numberOfRuns = 10 ;

WHILE @numberOfRuns > 0
    BEGIN;
        EXEC dbo.sp_WhoIsActive @get_transaction_info = 1, @get_plans = 1,
            @find_block_leaders = 1, @DESTINATION_TABLE = @destination_table ;

        SET @numberOfRuns = @numberOfRuns - 1 ;

        IF @numberOfRuns > 0
            BEGIN
                SET @msg = CONVERT(CHAR(19), GETDATE(), 121) + ': ' +
                 'Logged info. Waiting...'
                RAISERROR(@msg,0,0) WITH nowait ;

                WAITFOR DELAY '00:00:05'
            END
        ELSE
            BEGIN
                SET @msg = CONVERT(CHAR(19), GETDATE(), 121) + ': ' + 'Done.'
                RAISERROR(@msg,0,0) WITH nowait ;
            END

    END ;
GO

Step 3: Set Up Your Query to Look at the Results

This is optional, but since the table is named by datestamp, I like to keep a query on hand to look at the results without having to copy and paste the table name. I also like to put the collection_time column first and sort by it.

DECLARE @destination_table NVARCHAR(2000), @dSQL NVARCHAR(4000) ;
SET @destination_table = 'WhoIsActive_' + CONVERT(VARCHAR, GETDATE(), 112) ;
SET @dSQL = N'SELECT collection_time, * FROM dbo.' +
 QUOTENAME(@destination_table) + N' order by 1 desc' ;
print @dSQL
EXEC sp_executesql @dSQL
Comments { 17 }