Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New presentation - Backup Basics with Powershell and dbatools #16

Open
wants to merge 62 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
7bb1c11
changes
alevyinroc Jul 31, 2019
85b2bc1
Merge branch 'master' of https://github.com/alevyinroc/community-pres…
alevyinroc Jul 31, 2019
740b7c9
Rearranging to acommodate multiple presentations
alevyinroc Jan 4, 2020
4bdf69d
Starting presentation on backup management with dbatools
alevyinroc Jan 4, 2020
e643736
Starting framework
alevyinroc Jan 4, 2020
f9b9742
ppt shell
alevyinroc Feb 16, 2020
609b588
Merge branch 'master' of https://github.com/alevyinroc/community-pres…
alevyinroc Feb 16, 2020
f953837
Merge remote-tracking branch 'upstream/master'
alevyinroc Feb 16, 2020
84476a7
Notebook start
alevyinroc Feb 16, 2020
2e772c7
Making progress
alevyinroc Mar 14, 2020
110729c
Making progress
alevyinroc Nov 29, 2020
52b429b
Making good progress; checkpoint
alevyinroc Dec 5, 2020
73d5da4
Clear some output
alevyinroc Dec 5, 2020
648c021
Progress
alevyinroc Dec 5, 2020
1af731e
Progress
alevyinroc Dec 13, 2020
751065c
Tough run-through, work needed
alevyinroc Dec 13, 2020
65123f9
Add PIT and backup validation
alevyinroc Dec 20, 2020
6056d11
Moving things around
alevyinroc Dec 20, 2020
926990e
Another run-through
alevyinroc Dec 26, 2020
7270d72
More detail, rearranging
Dec 26, 2020
7d97013
Working on the deck
Dec 26, 2020
597de0c
Lots more words
alevyinroc Dec 27, 2020
0e897b5
more words
alevyinroc Dec 27, 2020
3d08259
Working on the PPT
alevyinroc Dec 27, 2020
0e50350
Rearrange, fix up a few things
alevyinroc Dec 31, 2020
2a4a372
Another run-through
alevyinroc Jan 2, 2021
247ae4c
More updates after runthroughs
alevyinroc Jan 3, 2021
9129e70
PIT restore demo finally fixed
alevyinroc Jan 4, 2021
f513288
Final runthrough
alevyinroc Jan 5, 2021
9539be5
ONDT session complete
alevyinroc Jan 6, 2021
bdc09bd
Renames
alevyinroc Jan 6, 2021
1908760
Renames complete
alevyinroc Jan 6, 2021
03e76a8
Post-KC User Group
alevyinroc Feb 20, 2021
81644c3
Merge branch 'master' of https://github.com/alevyinroc/community-pres…
alevyinroc Feb 20, 2021
0748fed
Breaking notebooks into modules
alevyinroc Aug 1, 2021
70712c6
Cleanup
alevyinroc Aug 1, 2021
eeb1c07
Gettign ready for Summit
alevyinroc Oct 11, 2021
61f582a
More Summit prep
alevyinroc Oct 13, 2021
8faa594
Switching to plain Powershell
alevyinroc Oct 14, 2021
2d80e84
Spelling corrections
alevyinroc Oct 14, 2021
de12f0c
Misc
alevyinroc Oct 14, 2021
2eb46a5
Workspace setup
alevyinroc Oct 14, 2021
0a169ed
Continuing the conversion
alevyinroc Oct 14, 2021
ed192ff
Finished migrating main scripts for Summit
alevyinroc Oct 15, 2021
b6d6338
Full run-through
alevyinroc Oct 20, 2021
806f34c
Minor tweaks
alevyinroc Oct 21, 2021
c4543f2
Cleaning up
alevyinroc Oct 21, 2021
85399a6
Shuffle things a bit
alevyinroc Oct 23, 2021
707e741
Migrate conventions to plain Powershell
alevyinroc Oct 23, 2021
21908ed
Add errorlog story
alevyinroc Oct 23, 2021
6589d9d
Move things a bit, add clipboard
alevyinroc Oct 23, 2021
dddeeff
Documentation change
alevyinroc Oct 23, 2021
1df3982
Slides
alevyinroc Oct 25, 2021
fba6519
Switch to shorter demo
alevyinroc Oct 25, 2021
5511a9a
Switch to running the log backup job
alevyinroc Oct 25, 2021
378f921
Switch from notebook to basic script
alevyinroc Nov 6, 2021
44ff6cf
Old slides
alevyinroc Nov 8, 2021
ec341c7
Links to resources
alevyinroc Dec 13, 2021
2a04a40
Formatting & switching to use prefix
alevyinroc Dec 13, 2021
fe83330
changing path for resored DB
alevyinroc Dec 13, 2021
d5e70ba
New version of presentation for UGs
alevyinroc Dec 13, 2021
97d03bd
Link to this repository!
alevyinroc Dec 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions andy-levy/BackupBasics/.vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"jupyter.jupyterServerType": "local"
}
10 changes: 10 additions & 0 deletions andy-levy/BackupBasics/00_Before_Starting.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<#
Do this before starting new session
#>
Clear-Host;
Remove-DbaDatabase -SqlInstance flexo\sql17 -database stackoverflow2010 -confirm:$false;
Remove-Item -force -Path 'C:\SQL\Backup\FLEXO$SQL17\StackOverflow2010' -recurse -confirm:$false;
Restore-DbaDatabase -SqlInstance FLEXO\sql17 -DatabaseName StackOverflow2010 -ReplaceDbNameInFile -WithReplace -Path C:\Datasets\StackOverflow2010.bak;
Invoke-DbaQuery -SqlInstance FLEXO\sql17 -Database msdb -Query "exec sp_delete_database_backuphistory @database_name='StackOverflow2010';";
Start-DbaAgentJob -SqlInstance FLEXO\sql17 -Job "DatabaseBackup - USER_DATABASES - FULL" -Wait;
Start-DbaAgentJob -SqlInstance FLEXO\sql17 -Job "DatabaseBackup - USER_DATABASES - Log" -Wait;
38 changes: 38 additions & 0 deletions andy-levy/BackupBasics/00_Why_Azure_Data_Studio.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"metadata": {
"kernelspec": {
"name": "SQL",
"display_name": "SQL",
"language": "sql"
},
"language_info": {
"name": "sql",
"version": ""
},
"extensions": {
"azuredatastudio": {
"version": 1,
"views": []
}
}
},
"nbformat_minor": 2,
"nbformat": 4,
"cells": [
{
"cell_type": "markdown",
"source": [
"# Why Azure Data Studio?\r\n",
"\r\n",
"With Azure Data Studio, we can mix text (in Markdown form), code, and the code's output, into a single package.\r\n",
"\r\n",
"We can then hand that package to someone else, ask them to run it, save the results, and send it back.\r\n",
"\r\n",
"This lets us keep the code nicely organized with the output, and intersperse instructions and other documentation."
],
"metadata": {
"azdata_cell_guid": "c5ce23b2-0603-47b4-9f4d-c63b228e2883"
}
}
]
}
23 changes: 23 additions & 0 deletions andy-levy/BackupBasics/01_Installation.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Clear-Host;
<#
Installation & Maintenance

Check for an existing dbatools module installation
#>
Get-Module -ListAvailable -Name dbatools;

<#
Trust the PowerShell Gallery
#>
Set-PSRepository -Name PSGallery -InstallationPolicy Trusted;

<#
Update the existing module or install if it's not there
#>

if (Get-Module -ListAvailable -Name dbatools) {
Update-Module -Name dbatools -Verbose;
}
else {
Install-Module -Name dbatools -Scope CurrentUser -Verbose;
}
39 changes: 39 additions & 0 deletions andy-levy/BackupBasics/02_Conventions.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
Clear-Host;
<#
# Conventions
## Getting Help
Every dbatools function has extensive comment-based help accessible via `Get-Help`. This help is also available at https://docs.dbatools.io/

`Find-DbaCommand <searchterm>` is your friend! Use this function to locate functions related to what you need to do.
#>
$BackupFunctions = Find-DbaCommand -Pattern backup;

$BackupFunctions.Count;

$BackupFunctions;

<#
## Naming Conventions

* dbatools follows standard Powershell function naming conventions
* All dbatools function names follow the convention `Verb-DbaNoun`
* Most `Get`s have a corresponding `Set`
* Functions with `Db` in their name _usually_ want to operate on the database level, not the whole instance.
#>
<#
## Splatting

Variable splatting is a method in Powershell where we can pass a collection of parameters to a function. This makes it easier to:
* Read without scrolling horizontally
* Dynamically change the parameter list passed to a function
For more information, see [`get-help about_splatting`](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_splatting?view=powershell-7.1)

When using splatting with switch parameters, you must explicitly state `$true` or `$false` for the switch value. For example, `-Verbose` becomes `Verbose = $true;`
#>

$BackupParams = @{
SqlInstance = "FLEXO\sql17";
Path = $BackupPath;
Database = "satellites";
CreateFolder = $true;
}
43 changes: 43 additions & 0 deletions andy-levy/BackupBasics/03_Instance_Objects.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
clear-host;
<#
# Instance-level Objects
## Keys & Certificates
### Functions Demonstrated
* `Backup-DbaServiceMasterKey`
* `Backup-DbaDbMasterKey`
* `Backup-DbaDbCertificate`

### Service Master Key
#>

$KeyBackupPassword = "MyP@$$w3rd" | ConvertTo-SecureString -AsPlainText -Force;
$KeyBackupParams = @{
SqlInstance = "FLEXO\sql17";
Path = "C:\SQL\Backup";
Password = $KeyBackupPassword;
};
Backup-DbaServiceMasterKey @KeyBackupParams;

<#
### Database Master Key
#>

$KeyBackupParams += @{
Database = "master";
};

Backup-DbaDbMasterKey @KeyBackupParams;

<#
### Certificates
#>
$CertBackupParams = @{
SqlInstance = "FLEXO\sql17";
Path = "C:\SQL\Backup";
EncryptionPassword = "My0th3rP@ssw0rD" | ConvertTo-SecureString -AsPlainText -Force;
DecryptionPassword = "bathrobe.rifleman.resent.demit" | ConvertTo-SecureString -AsPlainText -Force; # This is the password you set when creating the certificate
Database = "master";
Certificate = "TDECert_2021";
};

Backup-DbaDbCertificate @CertBackupParams;
114 changes: 114 additions & 0 deletions andy-levy/BackupBasics/04_Basic_Backup_Restore.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
Clear-Host;
<#
# Database Backup
## Basic Backup
### Function Demonstrated
* `Backup-DbaDatabase`
#>
$BackupPath = "C:\SQL\Backup\";
$BackupParams = @{
SqlInstance = "FLEXO\sql17";
Path = $BackupPath;
Database = "satellites";
CreateFolder = $true;
}
$BackupResult = Backup-DbaDatabase @BackupParams;

$BackupResult | Format-List -Property *;

# Save this for later
$SingleBackupFile = $BackupResult.BackupPath;


<#
Let's go a bit more complex
- Copy-only
- Multiple files
- Compression
- Checksum & Verify
- Custom timestamp format
- Adjust `MaxTransferSize` and `BufferCount`

See https://sirsql.net/2012/12/13/20121212automated-backup-tuning/ for scripts to test your own backup performance
#>
$BackupParams += @{
CopyOnly = $true;
Type = "Full";
FileCount = 4;
CompressBackup = $true;
Checksum = $true;
Verify = $true;
BufferCount = 1000;
MaxTransfersize = 2 * 1MB;
TimeStampFormat = "yyyy-MMM-dd HH.mm.ss";
}
$BackupResult = Backup-DbaDatabase @BackupParams;

$BackupResult | Format-List -Property *;

<#
What are we running?

The -OutputScriptOnly switch parameter tells -Backup-DbaDatabase to not perform the backup but instead show the T-SQL to execute the backup.
#>
$BackupParams += @{
OutputScriptOnly = $true;
}

Backup-DbaDatabase @BackupParams | Set-Clipboard;
<#

#>

<#
## Restoring the Latest Backup
* The `-Path` parameter specifies a path to search for backups. If multiple backups are found, the most recent one will be used.
* `-Database` is the name the database will have when restored, not the original name of the database when it was backed up.

### Function Demonstrated
* `Restore-DbaDatabase`
#>
Clear-Host;
$RestoreParams = @{
SqlInstance = "FLEXO\sql19";
Path = "C:\SQL\Backup\Satellites";
Database = "Satellites19";
}
$RestoreResult = Restore-DbaDatabase @RestoreParams;

$RestoreResult | Format-List -Property *;

<#
## Restoring a Specific Backup
If the database name we're restoring to already exists, `-WithReplace` will overwrite it. **Use with caution!**

### Function Demonstrated
* `Set-DbaDbOwner`
#>
$RestoreParams["Path"] = $SingleBackupFile;
$RestoreParams += @{
WithReplace = $true;
}
$RestoreResult = Restore-DbaDatabase @RestoreParams;

$RestoreResult | Format-List -Property *;

Set-DbaDbOwner -SqlInstance FLEXO\sql19 -Database Satellites19 -TargetLogin sa;

<#
## Just Looking!

Maybe I'm just looking to learn how to construct a `RESTORE DATABASE` SQL statement. Other times, I just want to review the statement before it's executed.

The `-ReplaceDbNameInFile` renames the pysical files to match the database name when restored.
#>

$RestoreParams += @{
OutputScriptOnly = $true;
ReplaceDbNameInFile = $true;
}

Restore-DbaDatabase @RestoreParams | Set-Clipboard;
<#

#>
67 changes: 67 additions & 0 deletions andy-levy/BackupBasics/05_Point_In_Time_Restore.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
Clear-Host;
<#
# Point In Time Restore
Who hasn't accidentally dropped a table?

## Function Demonstrated:
* `Invoke-DbaQuery`
#>

$PreUpdateTime = Get-Date;
$SOUpdateParams = @{
SqlInstance = "FLEXO\sql17";
Database = "StackOverflow2010";
Query = "drop table [Users];";
}
Invoke-DbaQuery @SOUpdateParams;

<#
Let's restore the database so we can fix the data

Take a log backup so we have something to work with
#>
Start-DbaAgentJob -SqlInstance FLEXO\sql17 -Job "DatabaseBackup - USER_DATABASES - Log" -Wait;

<#
Now we'll restore the database to a new one to use as a reference
#>
$RestoreParams = @{
SqlInstance = "FLEXO\sql17";
Path = 'C:\sql\Backup\FLEXO$SQL17\StackOverflow2010\';
DatabaseName = "StackOverflow2010";
RestoredDatabaseNamePrefix = "Restored";
RestoreTime = $PreUpdateTime;
ReplaceDbNameInFile = $true;
MaintenanceSolutionBackup = $true;
WithReplace = $true;
}
$RestoreResult = Restore-DbaDatabase @RestoreParams;

$RestoreResult | Format-List -Property *;

$RestoredDBName = $RestoreResult[-1].Database;

<#
Database is restored, let's verify the table is there
#>

$SOQueryParams = @{
SqlInstance = "FLEXO\sql17";
Database = "StackOverflow2010";
Query = "select count(*) as UserCount from [Users];"
}

Invoke-DbaQuery @SOQueryParams | Format-Table -AutoSize;

$SOQueryParams["Database"] = $RestoredDBName;
Invoke-DbaQuery @SOQueryParams | Format-Table -AutoSize;

<#
## Cleanup

Data looks good in the restored database, so we'll fix things up in the live database using that data (not shown), then remove the restored database.
## Function Demonstrated:
* `Remove-DbaDatabase`
#>

Remove-DbaDatabase -SqlInstance FLEXO\SQL17 -Database $RestoredDBName;
21 changes: 21 additions & 0 deletions andy-levy/BackupBasics/06_Backup_Setup.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Clear-Host;
<#
# Basic Backup Setup
## Where do backups go?
### Function Demonstrated
* `Get-DbaDefaultPath`
#>

Get-DbaDefaultPath -SqlInstance FLEXO\Sql17;

Get-DbaDefaultPath -SqlInstance FLEXO\Sql17,FLEXO\Sql19 | Select-Object ComputerName,ErrorLog;

<#
## Check & Set Backup Compression
### Functions Demonstrated
* `Get-DbaSpConfigure`
* `Set-DbaSpConfigure`
#>

Get-DbaSpConfigure -SqlInstance FLEXO\Sql17 -Name DefaultBackupCompression;
Set-DbaSpConfigure -SqlInstance FLEXO\Sql17 -Name DefaultBackupCompression -Value 1;
Loading