setup_graylog.md

Graylog

For the lastest installation instructions for this repo please see this blog post: IR TALES: THE QUEST FOR THE HOLY SIEM: GRAYLOG + AUDITD + OSQUERY

WARNING
This Ansible playbook will allocate half of the systems memory to Elasticsearch. For example, if a machine has 16GBs of memory, 8GBs of memory will be allocated to Elasticsearch.
WARNING

Init Ansible playbook

1. vim macos-workshop/ChooseYourSIEMAdventure/hosts.ini and add the Graylog server IP address under [graylog]
2. vim macos-workshop/ChooseYourSIEMAdventure/group_vars/all.yml and set:
3. base_domain – <external_domain - group_vars/all.yml>
4. timezone – OPTIONAL – The default timezone is UTC+0
5. siem_username – Ignore this setting
6. siem_password – Set the Graylog admin password
7. 
8. vim macos-workshop/ChooseYourSIEMAdventure/group_vars/graylog.yml and set:
9. hostname – Set the desired hostname for the server
10. graylog_version – Set the desired version of Graylog to use
11. beats_port – OPTIONAL – Set the port to ingest logs using BEAT clients
12. elastic_version – OPTIONAL – Set the desired version of Elasticsearch to use with Graylog – best to leave as default
13. mongo_version – OPTIONAL – Set the desired version of Mongo to use with Graylog – best to leave as default
14. mongo_admin_username – OPTIONAL – Set Mongo admin username – best to leave as default
15. mongo_admin_password – Set the Mongo admin user password
16. mongo_graylog_username – Set Mongo username for Graylog user
17. mongo_graylog_password – Set Mongo password for Graylog user
18. 

Run Ansible playbook

1. ansible-playbook -i hosts.ini deploy_graylog.yml -u ubuntu
2. 

Generate Let's Encrypt certificate

1. SSH into Elastic EC2 instance
2. sudo su
3. apt install certbot python3-certbot-nginx -y
4. certbot --nginx -d graylog.<external domain>
5. Enter the e-mail for the admin of the domain
6. Enter A for Terms of Service
7. Enter N to share e-mail with EFF
8. Enter 2 to redirect HTTP traffic to HTTPS
9. Review NGINX config: /etc/nginx/conf.d/graylog.conf
10. systemctl restart nginx

Create workshop user

1. Open browser to https://graylog.<external domain> and login
2. Enter admin for username
3. Enter <siem_password> into Password
4. System > Users and Teams
5. Select "Create User" in top right
6. Enter threathunter into username
7. Enter threathunter into full name
8. Leave e-mail blank
9. Enter 6 for user session timeout
10. Enter a password for the user
11. Select User inspector for roles
12. 
13. Select "Create user"

Create Graylog indexes and streams

Create indexes

1. System > Indicies
2. Select "Create index set" in the top right
3. Enter Osquery for name
4. Enter Osquery logs for description
5. Enter osquery for index prefix
6. Leave everything as default
7. 
8. Select "Save"
9. Repeat the steps above to create an index called "test" and "sysmon

Create streams

1. Select "Streams" at the top
2. Select "Create stream" in the top right
3. Enter Osquery-stream for name
4. Enter Osquery stream for description
5. Select "Osquery" for index set
6. Check "Remove matches from 'All messages' stream"
7. 
8. Select "Save"
9. Repeat the steps above to create an index called "test"
10. Select "Manage Rules" for Osquery stream
11. Select “Add stream rule” on the right
12. Enter event_module for field
13. Select match exactly for type
14. Enter osquery for value
15. 
16. Select "Save"
17. 
18. Select "I’m done" in bottom left
19. Select "Start stream" for Osquery stream
20. Repeat the steps above for test

Create input for Kafka

1. System > Inputs
2. Select "Raw/Plaintext Kafka" from Input drop down
3. Select "Launch new input"
4. Enter Kafka input into Title
5. UNcheck Legacy Mode
6. Enter 172.16.50.10:9092 into Bootstrap servers
7. Enter 127.0.0.1:2181 into Zookeeper address
8. Enter osquery into topic filter regex
9. 
10. Select "Save"
11. Repeat the steps above to create an index called "test" and "sysmon

References