Skip to content
This repository has been archived by the owner on Jul 11, 2024. It is now read-only.

Commit

Permalink
Merge branch 'master' into site-redesign
Browse files Browse the repository at this point in the history
  • Loading branch information
heyitsols authored Apr 20, 2021
2 parents 471f67b + 5837584 commit 1016ad7
Show file tree
Hide file tree
Showing 5 changed files with 86 additions and 67 deletions.
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
language: ruby
rvm:
- 2.2
script: bundle exec rspec spec
- 2.7
script: bundle exec rspec spec
145 changes: 81 additions & 64 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
GEM
remote: https://rubygems.org/
specs:
activesupport (6.0.3.1)
activesupport (6.0.3.6)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
Expand All @@ -19,47 +19,50 @@ GEM
colorator (1.1.0)
commonmarker (0.17.13)
ruby-enum (~> 0.5)
concurrent-ruby (1.1.6)
diff-lcs (1.3)
dnsruby (1.61.3)
addressable (~> 2.5)
em-websocket (0.5.1)
concurrent-ruby (1.1.8)
diff-lcs (1.4.4)
dnsruby (1.61.5)
simpleidn (~> 0.1)
em-websocket (0.5.2)
eventmachine (>= 0.12.9)
http_parser.rb (~> 0.6.0)
ethon (0.12.0)
ffi (>= 1.3.0)
eventmachine (1.2.7)
execjs (2.7.0)
faraday (1.0.0)
faraday (1.3.0)
faraday-net_http (~> 1.0)
multipart-post (>= 1.2, < 3)
ffi (1.12.2)
ruby2_keywords
faraday-net_http (1.0.1)
ffi (1.15.0)
forwardable-extended (2.6.0)
gemoji (3.0.1)
github-pages (204)
github-pages-health-check (= 1.16.1)
jekyll (= 3.8.5)
github-pages (214)
github-pages-health-check (= 1.17.0)
jekyll (= 3.9.0)
jekyll-avatar (= 0.7.0)
jekyll-coffeescript (= 1.1.1)
jekyll-commonmark-ghpages (= 0.1.6)
jekyll-default-layout (= 0.1.4)
jekyll-feed (= 0.13.0)
jekyll-feed (= 0.15.1)
jekyll-gist (= 1.5.0)
jekyll-github-metadata (= 2.13.0)
jekyll-mentions (= 1.5.1)
jekyll-mentions (= 1.6.0)
jekyll-optional-front-matter (= 0.3.2)
jekyll-paginate (= 1.1.0)
jekyll-readme-index (= 0.3.0)
jekyll-redirect-from (= 0.15.0)
jekyll-redirect-from (= 0.16.0)
jekyll-relative-links (= 0.6.1)
jekyll-remote-theme (= 0.4.1)
jekyll-remote-theme (= 0.4.3)
jekyll-sass-converter (= 1.5.2)
jekyll-seo-tag (= 2.6.1)
jekyll-seo-tag (= 2.7.1)
jekyll-sitemap (= 1.4.0)
jekyll-swiss (= 1.0.0)
jekyll-theme-architect (= 0.1.1)
jekyll-theme-cayman (= 0.1.1)
jekyll-theme-dinky (= 0.1.1)
jekyll-theme-hacker (= 0.1.1)
jekyll-theme-hacker (= 0.1.2)
jekyll-theme-leap-day (= 0.1.1)
jekyll-theme-merlot (= 0.1.1)
jekyll-theme-midnight (= 0.1.1)
Expand All @@ -70,34 +73,35 @@ GEM
jekyll-theme-tactile (= 0.1.1)
jekyll-theme-time-machine (= 0.1.1)
jekyll-titles-from-headings (= 0.5.3)
jemoji (= 0.11.1)
kramdown (= 1.17.0)
jemoji (= 0.12.0)
kramdown (= 2.3.1)
kramdown-parser-gfm (= 1.1.0)
liquid (= 4.0.3)
mercenary (~> 0.3)
minima (= 2.5.1)
nokogiri (>= 1.10.4, < 2.0)
rouge (= 3.13.0)
rouge (= 3.26.0)
terminal-table (~> 1.4)
github-pages-health-check (1.16.1)
github-pages-health-check (1.17.0)
addressable (~> 2.3)
dnsruby (~> 1.60)
octokit (~> 4.0)
public_suffix (~> 3.0)
public_suffix (>= 2.0.2, < 5.0)
typhoeus (~> 1.3)
html-pipeline (2.12.3)
html-pipeline (2.14.0)
activesupport (>= 2)
nokogiri (>= 1.4)
http_parser.rb (0.6.0)
i18n (0.9.5)
concurrent-ruby (~> 1.0)
jekyll (3.8.5)
jekyll (3.9.0)
addressable (~> 2.4)
colorator (~> 1.0)
em-websocket (~> 0.5)
i18n (~> 0.7)
jekyll-sass-converter (~> 1.0)
jekyll-watch (~> 2.0)
kramdown (~> 1.14)
kramdown (>= 1.17, < 3)
liquid (~> 4.0)
mercenary (~> 0.3.3)
pathutil (~> 0.9)
Expand All @@ -117,33 +121,34 @@ GEM
rouge (>= 2.0, < 4.0)
jekyll-default-layout (0.1.4)
jekyll (~> 3.0)
jekyll-feed (0.13.0)
jekyll-feed (0.15.1)
jekyll (>= 3.7, < 5.0)
jekyll-gist (1.5.0)
octokit (~> 4.2)
jekyll-github-metadata (2.13.0)
jekyll (>= 3.4, < 5.0)
octokit (~> 4.0, != 4.4.0)
jekyll-mentions (1.5.1)
jekyll-mentions (1.6.0)
html-pipeline (~> 2.3)
jekyll (>= 3.7, < 5.0)
jekyll-optional-front-matter (0.3.2)
jekyll (>= 3.0, < 5.0)
jekyll-paginate (1.1.0)
jekyll-readme-index (0.3.0)
jekyll (>= 3.0, < 5.0)
jekyll-redirect-from (0.15.0)
jekyll-redirect-from (0.16.0)
jekyll (>= 3.3, < 5.0)
jekyll-relative-links (0.6.1)
jekyll (>= 3.3, < 5.0)
jekyll-remote-theme (0.4.1)
jekyll-remote-theme (0.4.3)
addressable (~> 2.0)
jekyll (>= 3.5, < 5.0)
rubyzip (>= 1.3.0)
jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0)
rubyzip (>= 1.3.0, < 3.0)
jekyll-sass-converter (1.5.2)
sass (~> 3.4)
jekyll-seo-tag (2.6.1)
jekyll (>= 3.3, < 5.0)
jekyll-seo-tag (2.7.1)
jekyll (>= 3.8, < 5.0)
jekyll-sitemap (1.4.0)
jekyll (>= 3.7, < 5.0)
jekyll-swiss (1.0.0)
Expand All @@ -156,8 +161,8 @@ GEM
jekyll-theme-dinky (0.1.1)
jekyll (~> 3.5)
jekyll-seo-tag (~> 2.0)
jekyll-theme-hacker (0.1.1)
jekyll (~> 3.5)
jekyll-theme-hacker (0.1.2)
jekyll (> 3.5, < 5.0)
jekyll-seo-tag (~> 2.0)
jekyll-theme-leap-day (0.1.1)
jekyll (~> 3.5)
Expand Down Expand Up @@ -191,53 +196,60 @@ GEM
jekyll (>= 3.3, < 5.0)
jekyll-watch (2.2.1)
listen (~> 3.0)
jemoji (0.11.1)
jemoji (0.12.0)
gemoji (~> 3.0)
html-pipeline (~> 2.2)
jekyll (>= 3.0, < 5.0)
kramdown (1.17.0)
kramdown (2.3.1)
rexml
kramdown-parser-gfm (1.1.0)
kramdown (~> 2.0)
liquid (4.0.3)
listen (3.2.1)
listen (3.5.1)
rb-fsevent (~> 0.10, >= 0.10.3)
rb-inotify (~> 0.9, >= 0.9.10)
mercenary (0.3.6)
mini_portile2 (2.4.0)
mini_portile2 (2.5.0)
minima (2.5.1)
jekyll (>= 3.5, < 5.0)
jekyll-feed (~> 0.9)
jekyll-seo-tag (~> 2.1)
minitest (5.14.1)
minitest (5.14.4)
multipart-post (2.1.1)
nokogiri (1.10.8)
mini_portile2 (~> 2.4.0)
octokit (4.16.0)
nokogiri (1.11.2)
mini_portile2 (~> 2.5.0)
racc (~> 1.4)
octokit (4.20.0)
faraday (>= 0.9)
sawyer (~> 0.8.0, >= 0.5.3)
pathutil (0.16.2)
forwardable-extended (~> 2.6)
public_suffix (3.1.1)
rake (13.0.1)
rb-fsevent (0.10.3)
public_suffix (4.0.6)
racc (1.5.2)
rake (13.0.3)
rb-fsevent (0.10.4)
rb-inotify (0.10.1)
ffi (~> 1.0)
rexml (3.2.4)
robotex (1.0.0)
rouge (3.13.0)
rspec (3.9.0)
rspec-core (~> 3.9.0)
rspec-expectations (~> 3.9.0)
rspec-mocks (~> 3.9.0)
rspec-core (3.9.1)
rspec-support (~> 3.9.1)
rspec-expectations (3.9.0)
rouge (3.26.0)
rspec (3.10.0)
rspec-core (~> 3.10.0)
rspec-expectations (~> 3.10.0)
rspec-mocks (~> 3.10.0)
rspec-core (3.10.1)
rspec-support (~> 3.10.0)
rspec-expectations (3.10.1)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.9.0)
rspec-mocks (3.9.1)
rspec-support (~> 3.10.0)
rspec-mocks (3.10.2)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.9.0)
rspec-support (3.9.2)
ruby-enum (0.7.2)
rspec-support (~> 3.10.0)
rspec-support (3.10.2)
ruby-enum (0.9.0)
i18n
rubyzip (1.3.0)
ruby2_keywords (0.0.4)
rubyzip (2.3.0)
safe_yaml (1.0.5)
sass (3.7.4)
sass-listen (~> 4.0.0)
Expand All @@ -247,15 +259,20 @@ GEM
sawyer (0.8.2)
addressable (>= 2.3.5)
faraday (> 0.8, < 2.0)
simpleidn (0.2.1)
unf (~> 0.1.4)
terminal-table (1.8.0)
unicode-display_width (~> 1.1, >= 1.1.1)
thread_safe (0.3.6)
typhoeus (1.3.1)
typhoeus (1.4.0)
ethon (>= 0.9.0)
tzinfo (1.2.7)
tzinfo (1.2.9)
thread_safe (~> 0.1)
unicode-display_width (1.6.1)
zeitwerk (2.3.0)
unf (0.1.4)
unf_ext
unf_ext (0.0.7.7)
unicode-display_width (1.7.0)
zeitwerk (2.4.2)

PLATFORMS
ruby
Expand All @@ -267,4 +284,4 @@ DEPENDENCIES
rspec

BUNDLED WITH
2.0.1
2.0.2
2 changes: 1 addition & 1 deletion _includes/footer.html
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,4 @@ <h3>Before you go&hellip;</h3>
</footer>
</div>
</body>
</html>
</html>
2 changes: 2 additions & 0 deletions _posts/2020-12-09-its-just-a-monitoring-change.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,5 @@ As with all major incidents and service interruptions, a Post Incident Review wa
A number of these improvements are already underway, and changes have been made to the `query_exporter` to ensure that it doesn't ever attempt to create more than two simultaneous connections to the database. As for the underlying cause of the incident (or the "root cause" if you insist on using such language), that has to be the fact that our assumptions as teams or individuals are ultimately formed by our past experiences. Given our past experiences would suggest that a metrics endpoint is relatively low-resource, we saw no problem with polling it as such a high rate. Engineers that work directly with the database, and have indeed already written their own exporters that make use of database queries, would know for certain that this should be handled with a lot more caution.

Above all else, this has been a very useful reminder that even what appears to be a simple change to monitoring might in fact be the thing that causes your revenue-generating services to grind to a halt.

**EDIT (2021-02-22): The contents of this blog post have been discussed further on the [Software Misadventures](https://softwaremisadventures.com/podcast/2021/02/ols/) podcast.**
Binary file removed images/logos/logo-cross-sky-lotto.png
Binary file not shown.

0 comments on commit 1016ad7

Please sign in to comment.