Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

Practical Log Collection for Security Operations Using Elastic Stack

Tech May 26 6

Effective incident response and attack attribution rely heavily on comprehensive log data. To detect, analyze, and block adversaries early in their lifecycle, organizations must collect diverse logs across endpoints and infrastructure.

Deploying Elasticsearch and Kibana

RPM-Based Installation (Recommended for Stability)

Due to instability observed with Docker deployments, a native RPM installation is preferred:

# Download and verify Elasticsearch
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.12.1-x86_64.rpm
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.12.1-x86_64.rpm.sha512
shasum -a 512 -c elasticsearch-8.12.1-x86_64.rpm.sha512
sudo rpm -i elasticsearch-8.12.1-x86_64.rpm

# Download and verify Kibana
wget https://artifacts.elastic.co/downloads/kibana/kibana-8.12.1-x86_64.rpm
wget https://artifacts.elastic.co/downloads/kibana/kibana-8.12.1-x86_64.rpm.sha512
shasum -a 512 -c kibana-8.12.1-x86_64.rpm.sha512
sudo rpm -i kibana-8.12.1-x86_64.rpm

After installation, configure both services to bind to all interfaces:

/etc/elasticsearch/elasticsearch.yml

http.host: 0.0.0.0
transport.host: 0.0.0.0

/etc/kibana/kibana.yml

server.host: "0.0.0.0"

Enable and start the services:

sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch.service
sudo systemctl enable --now kibana.service

Verify Elasticsearch connectivity:

export ELASTIC_PASSWORD="<generated_password>"
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200

Setting Up Fleet and Elastic Agent

In Kibana, navigate to Management → Fleet and deploy a Fleet Server (typically on port 8220 over HTTPS). Note: Fleet Server and Elastic Agent cannot coexist on the same host.

Once the Fleet Server is active, enroll additional Elastic Agents. To enhance security visibility, install integrations via Security → Manage → Add security integrations, enabling detection rules such as reverse shell monitoring.

Ensure detection rules are enabled under Security → Rules to populate the Detection & Response dashboard.

Collecting Windows Event Logs with Winlogbeat

Windows event logs require structured collection. Winlogbeat is used to forward logs to Elasticsearch.

Sample winlogbeat.yml configuration:

winlogbeat.event_logs:
  - name: Application
    ignore_older: 72h
  - name: System
  - name: Security
  - name: Microsoft-Windows-Sysmon/Operational
  - name: Windows PowerShell
    event_id: 400, 403, 600, 800
  - name: Microsoft-Windows-PowerShell/Operational
    event_id: 4103, 4104, 4105, 4106
  - name: ForwardedEvents
    tags: [forwarded]

setup.kibana:
  host: "kibana_host:5601"

output.elasticsearch:
  hosts: ["elasticsearch_host:9200"]
  username: "elastic"
  password: "<password>"
  pipeline: "winlogbeat-%{[agent.version]}-routing"

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~

Validate the configuration:

./winlogbeat.exe test config -c ./winlogbeat.yml -e

Start the service and verify data ingestion by listing Elasticsearch indices:

curl http://elasticsearch_host:9200/_cat/indices?v | grep winlogbeat

If data appears in Elasticsearch but not in Kibana, manually create an index pattern:

  1. Go to Discover → Settings (gear icon) → Index Patterns
  2. Create a new pattern matching winlogbeat-*
  3. Set @timestamp as the time field

Data should now be queryable in Discover, enabling analysis of Security, Sysmon, and PowerShell events for threat hunting and forensic investigations.

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.