You’ll want to open an ssh session to complete this.
By Joe Piggee
|
Commands
|
|
|
|
|
|
|
|
|
|
Restart ESM
|
|
|
|
|
# service cpservice stop
|
|
|
|
|
# service cpservice start
|
|
|
|
|
|
|
|
|
|
Restart APM
|
|
|
|
|
# /etc/init.d/apm stop
|
|
|
|
|
# /etc/init.d/apm start
|
|
|
|
|
# NitroStop --nod (no daemon)
|
|
|
|
|
# NitroStart --nod (no daemon)
|
|
|
|
|
|
|
|
|
|
View ESM DB Errors
|
|
|
|
|
tailf /usr/local/ess/data/NitroError.Log
|
|
|
|
|
less /usr/local/ess/data/NitroError.Log (Use
‘q’ to exit the text editor)
|
|
|
|
|
|
|
|
|
|
View Rebuild Status of DB Partitions (in
addition to ESM System Properties page)
|
|
|
|
|
# /data_hd/usr/local/ess/data/watch –d ‘ls
–lia *X*’
|
|
|
|
|
|
|
|
|
|
Viewing Sort Files created by ESM (too large
can create slowdowns)
|
|
|
|
|
# /data_hd/usr/local/ess/data/du –ch *SRT*
|
|
|
|
|
|
|
|
|
|
Retrieve APM Crash Logs Location
|
|
|
|
|
# /var/log/ice/crash.log
|
|
|
|
|
|
|
|
|
|
Retrieve Health Status on SIEM device
|
|
|
|
|
# cat /var/lib/HealthStatus.data (tac for
newest first, cat for oldest first)
|
|
|
|
|
|
|
|
|
|
Gather appliance resource information
|
|
|
|
|
# grep Resource /var/log/messages
|
|
|
|
|
|
|
|
|
|
Check Event Receiver for Restart Events
|
|
|
|
|
cat /var/log/messages* | grep -i
"starting ERC" (tac for newest first, cat for oldest first)
|
|
|
|
|
less /var/log/messages (type ‘q’ to exit the
editor, “Shift + g” to get to the end of the file)
|
|
|
|
|
|
|
|
|
|
Check upgrade history
|
|
|
|
|
# cat /etc/upgrade.history (tac for newest
first, cat for oldest first)
|
|
|
|
|
|
|
|
|
|
Zip Messages Log
|
|
|
|
|
# cd /var/log
|
|
|
|
|
# tar –czvf messages.tgz messages*
|
|
|
|
|
|
|
|
|
|
Checking RAID drives (from
/opt/MegaRAID/MegaCli/)
|
|
|
|
|
#./MegaCli64 -ldinfo -l0 -a0
|
|
|
|
|
#./MegaCli64 -ldinfo –l1 -a0
|
|
|
|
|
#./MegaCli64 -pdlist -a0
|
|
|
|
|
|
|
|
|
|
Most helping CPU processes view
|
|
|
|
|
# ps -ejH (shows processes and threads)
|
|
|
|
|
# ps -C commandname (search like "ps |
grep")# ps -auxf (most details)
|
|
|
|
|
# ps -ax (all processes)
|
|
|
|
|
|
|
|
|
|
View the Build Version of the McAfee
Software on the appliance
|
|
|
|
|
# cat /etc/buildstamp
|
|
|
|
|
|
|
|
|
|
View the build version of the Linux OS
|
|
|
|
|
# cat /proc/version
|
|
|
|
|
|
|
|
|
|
Check Disk Space – General
|
|
|
|
|
# df –h (to show space by drive) or du –h
(to show space by folder)
|
|
|
|
|
|
|
|
|
|
Check if logs are being sent to a receiver
|
|
|
|
|
# tcpdump –nni eth1 host 10.x.x.x
|
|
|
|
|
Check multiple hosts
|
|
|
|
|
# tcpdump –nni eth1 ‘(host 10.x.x.x or host
10.y.y.y)’
|
|
|
|
|
|
|
|
|
|
Finding the largest file on a system
|
|
|
|
|
Use the du command to find out how much data
is in each directory.
|
|
|
|
|
# du -hc --max-depth=1
|
|
|
|
|
|
|
|
|
|
This will return the base level directories
with the size.
|
|
|
|
|
NOTE: is best to start in the "/"
directory.
|
|
|
|
|
|
|
|
|
|
The following command will limit the results
to only the Gigabyte size directories:
|
|
|
|
|
# du -hc --max-depth=1 | grep G
|
|
|
|
|
You can continue to use
these commands as you drill down in the directories to find the directory
that contains the largest amount of data.
|
|
|
|
|
Check the disk utilization
|
|
|
|
|
Sar –d 1 10
|
|
|
|
|
Checks disk utilization
every second over 10 seconds
|
|
|
|
|
|
|
|
|
|
TCP dump commands when you do not see the
expected DBM data
|
|
|
|
|
# ifconfig -a (determines which eth
port DB traffic is being sent to)
|
|
|
|
|
# tcpdump -s0 -ieth3 -wfile1.pcap host
1.2.3.4 and port 1433 (dump file will be called file1.pcap in the current
directory)
|
|
|
|
|
Run the dump for 30,
and then press CTRL+C to escape out. Check the file size and repeat. Generally
a few hundred MBs worth of files should be good if you see the expected DB
traffic in the tcpdump.
|
|
|
|
|
|
|
|
|
|
To check for vlan tagging
|
|
|
|
|
# tcpdump -s0 -ieth3 -wfile1.pcap vlan
# host 10.x.x.x and port 1433 (optional to leave the tag number, for example
130, off if unknown).
|
|
|
|
|
# tcpdump -s0 -nnvXi eth1 vlan # and
host 172.x.x.x and port 1433 (with vlan tag and shows packet contents in list
form)
|
|
|
|
|
# tcpdump -s0 -nnvXi eth1 -wfile3.pcap
vlan # and host 172.x.x.x and port 1433 (with vlan tag and saves
tcpdump to pcap file)
|
|
|
|
|
|
|
|
|
|
Find all Receiver database index files
|
|
|
|
|
# ls /var/log/data/inline/
|
|
|
|
|
|
|
|
|
|
Track Receiver database index progress
|
|
|
|
|
# tailf
/var/log/data/inline/NitroError.Log
|
|
|
|
|
|
|
|
|
|
Restarting collectors /
parsers on a Receiver
|
|
|
|
|
# killall collectorsctl filterctl
parsersctl
|
|
|
|
|
# /etc/init.d/nitrodbserver restart
|
|
|
|
|
# collectorsctl -- +laux
|
|
|
|
|
# filterctl -- +laux
|
|
|
|
|
# parsersctl -- +laux
|
|
|
|
|
Restarting WMI
collector / parser only
|
|
|
|
|
# killall -9 wmin
|
|
|
|
|
# killall -9 wmip
|
|
|
|
|
Run checksum on install
file
|
|
|
|
|
# sha1sum
<filename>
|
|
|
|
|
Check DAS Virtual Drive
Information from ESM or ELM
|
|
|
|
|
Watch –d megacli –ldinfo
–l0 –a0
|
|
|
|
|
“Ctrl + C” to exit
|
|
|
|
|
Check DAS Drive Status
|
|
|
|
|
megacli -pdlist -a0 |
grep Firmware
|
|
|
|
|
Check the Controller,
RAID, and Drive Status on ESM
|
|
|
|
|
Show the Controller:
|
|
|
|
|
# tw_cli show
|
|
|
|
|
Show the Status of the
RAID/Drives
|
|
|
|
|
# tw_cli show c# (# shown using show command – c2 for ESM’s, c0 on 2250
Receivers and APM)
|
|
|
|
|
Show individual drive
status
|
|
|
|
|
# tw_cli info c2 p# (# drive number)
|
|
|
|
|
Show individual drive
serial number
|
|
|
|
|
# tw_cli info c2 p# serial (# drive number)
|
|
|
|
|
Check HA or Cluster
Status on Receiver or ESM
|
|
|
|
|
# ha_status
|
|
|
|
|
# crm status
|
|
|
|
|
Check agent logs status
on Receiver
|
|
|
|
|
# tailf
/var/log/NPP_c.log
|
|
|
|
|
# tailf
/var/log/NPP_p.log
|
|
|
|
|
Finding and viewing raw
logs in stored text file on Receiver (example)
|
|
|
|
|
# tail –n 1000
/var/log/NPP_c.log | grep 10.198.12.73
|
|
|
|
|
Results: NPP_c[18094]:
Got a connection from client IP addr: 10.198.12.73, id = 105
|
|
|
|
|
Find the ID of the host
and use on the next command to find files to search
|
|
|
|
|
# ls
/var/log/data/inline/thirdparty.logs/105/in/
|
|
|
|
|
# cat
/var/log/data/inline/thirdparty.logs/105/in/data.20121115161524000
|
|
|
|
|
View ESM DB partitions
|
|
|
|
|
# !nsq (shortcut command
which will launch the nsql command below – if in the correct directory)
|
|
|
|
|
# nsql
/usr/local/ess/data/connect_esm.sql (To exit the nsql editor type ‘x’ or
‘exit’ and <enter>)
|
|
|
|
|
(Attached = Good,
Detached = Call Support, Bad = Awaiting Rebuild)
|
|
|
|
|
To view Event data
|
|
|
|
|
show partitions from
alert
|
|
|
|
|
To view Flow data
|
|
|
|
|
show partitions from
connection
|
|
|
|
|
To view Appliance Packet
data
|
|
|
|
|
Show partitions from
packet
|
|
|
|
|
View record counts in DB
|
|
|
|
|
# nsql
/usr/local/ess/data/connect_esm.sql (To exit the nsql editor type ‘x’
or ‘exit’ and <enter>)
|
|
|
|
|
To view event data
|
|
|
|
|
select count(*) from
alert
|
|
|
|
|
To view flow data
|
|
|
|
|
select count(*) from
connection
|
|
|
|
|
To view Appliance Packet
data
|
|
|
|
|
select count(*) from
packet
|
|
|
|
|
View Event data on a
receiver
|
|
|
|
|
# nsql
/var/log/data/inline/connect.sql (Launch this command when ssh in to a
receiver)
|
|
|
|
|
To view Event data
|
|
|
|
|
show partitions from
event
|
|
|
|
|
To get the clutter off of
the SSH session and your prompt back to the top of the screen (does not
remove scroll back data on screen)
|
|
|
|
|
clear
|
|
|
|
|
Article for rekeying a
SIEM Appliance
|
|
|
|
|
# cd /root/.ssh
|
|
|
|
|
# cat
/etc/NitroGuard/factory-id_rsa.pub >authorized_keys2
|
|
|
|
|
Wiping Receiver Data
Sources and log files
|
|
|
|
|
# rm
/etc/NitroGuard/thirdparty.*.*
|
|
|
|
|
# rm
/var/log/data/inline/thirdparty.logs/(1* thru 9*)
|
|
|
|
|
# rm
/var/log/data/inline/thirdparty.logs/elm.logs/(1* thru 9*)
|
|
|
|
|
# rm
/var/log/data/inline/thirdparty.logs/elm.logs/tmp/(1* thru 9*)
|
|
|
|
|
ESM Related
|
|
|
|
|
Quickest running filters – Very
Important!!!!
|
|
|
|
|
There are combinations of filters that are
specifically tuned to run more quickly. These combinations have been
defined by users who frequently use the filters for quickly drilling down to
specific events.
|
|
|
|
|
|
|
|
|
|
Signature ID + Event
Subtype + Protocol
|
|
|
|
|
Signature ID + Source IP
|
|
|
|
|
Signature ID +
Destination IP
|
|
|
|
|
Source IP + Destination
IP
|
|
|
|
|
|
|
|
|
|
Additional Enhancements since 8.x:
|
|
|
|
|
Normalization ID + Source
IP
|
|
|
|
|
Normalization ID +
Destination IP
|
|
|
|
|
Normalization ID + Event
Subtype
|
|
|
|
|
Normalization ID +
Protocol
|
|
|
|
|
Normalization ID +
Signature ID
|
|
|
|
|
Source Zone + Destination
Zone
|
|
|
|
|
Directory Related
|
|
|
|
|
|
|
|
|
|
ESM & Receiver
Software File Location (for upgrades)
|
|
|
|
|
/usr/local/ess/SoftwareUpdates/
|
|
|
|
|
/usr/local/NitroGuard
|
|
|
|
|
|
|
|
|
|
ESM Software File
Locations Archives
|
|
|
|
|
/usr/local/ess/update/archive/
|
|
|
|
|
|
|
|
|
|
ESM Default Backup
Location
|
|
|
|
|
/db1/usr/local/ess/dbbackup/
|
|
|
|
|
|
|
|
|
|
ESM DB Blob & Index
Location
|
|
|
|
|
/data_hd/usr/local/ess/data/
|
|
|
|
|
|
|
|
|
|
ESM Index HD Location
(Most recent data)
|
|
|
|
|
/index_hd/usr/local/ess/data/
|
|
|
|
|
|
|
|
|
|
ESM Redundant File Copy
Location (For Alert, Connection, and Log files)
|
|
|
|
|
/usr/local/ess/dbredund
|
|
|
|
|
Commands:
McAfee-ETM-6000 ~ # less
/usr/local/ess/data/NitroError.Log
McAfee-ETM-6000 ~ # service cpservice
stop
tailf /var/log/messages - Watch
what's happening
Tcpdump -nni eth0 host <ipaddress
of host you want to get a dump from> -vvv -w <path to write dump>
NitroTID - get database
troubleshooting information
Indications that database needs to be
rebuilt:
2014/04/09 21:43:01.429 Error 132
opening table with field Alert.ALERTTIM(partition 2406)(data count =
77427336, index count = 77427320) Index count does not match record count
Rebuild Database index:
McAfee-ETM-6000 ~ # cd
/usr/local/ess/data
McAfee-ETM-6000 /usr/local/ess/data #
mkdir copy_ngcp
McAfee-ETM-6000 /usr/local/ess/data #
cp ngcp.cfg copy_ngcp
McAfee-ETM-6000 /usr/local/ess/data #
cp ngcp.cfd copy_ngcp
McAfee-ETM-6000 /usr/local/ess/data #
cd copy_ngcp/
McAfee-ETM-6000
/usr/local/ess/data/copy_ngcp # ls
ngcp.cfd ngcp.cfg
McAfee-ETM-6000
/usr/local/ess/data/copy_ngcp # cd ../
McAfee-ETM-6000 /usr/local/ess/data #
rm ngcp.cfg
McAfee-ETM-6000 /usr/local/ess/data #
rm ngcp.cfd
McAfee-ETM-6000 /usr/local/ess/data #
ls
Check Database:
McAfee-ETM-6000 ~ # service cpservice
stop
Stopping ESS cpservice
[ OK ]
McAfee-ETM-6000 ~ # DBCheck -d
'/usr/local/ess/data/ngcp.dfl' -p 'LOCDB327|CPDB126' -t
'!Alert|!Connection|!Log|!Packet|!stringmap' -r
McAfee-ETM-6000 ~ # DBCheck -d
'/usr/local/ess/data/ngcp.dfl' -p 'LOCDB327|CPDB126' -t -c
Option at position 5 needs an
argument : t
McAfee-ETM-6000 ~ # DBCheck -d
'/usr/local/ess/data/ngcp.dfl' -p 'LOCDB327|CPDB126' -c
DBCheck -d '/usr/local/ess/data/ngcp.dfl'
-p 'LOCDB327|CPDB126' -c | grep Not
|
|
|
|
|
Check database health:
1. Cd /usr/local/ess/data
2. DBCheck -d ngcp.dfl -c | grep not
|
|
|
|
|
Check Syslog for Raid errors:
1. Cd /usr/local/ess/data
2. Dmesg | grep RAID
|
|
|
|
|
Display Raid Status
1. Cd /usr/local/ess/data
2. MegaCli64 -CfgDsply -aAll |
grep fail or grep down
|
Use grep for "downgraded, or
fail"which will indicate raid issues
|
|
|
