"No Data" on most of the reports

[Mbutler522010]

I inherited a Fortigate 800C and FortiAnalyzer 100B - and I am pretty sure the Analyzer is not working right.

On the Fortigate, the "Send Logs to FortiAnalyzer" is checked, the IP Address is right, test connectivity shows all is ok. "Enable all" is checked for event logging

On the Analyzer, under Devices it shows the Fortigate Unit, has check marks for all permissions and shows "Data was received on 2015-01-12" and 8gb of logs are in use. In the summary list of devices, the "logs" column shows a green light.

On the Analyzer, When I go into "Log and Archive" and select "Traffic Log" I see screens of traffic events

But I don't seem to get anything. When I go to reports-Bandwidth and App Usage: "Top Users by Sessions" and "Top applications by sessions" have bar charts, but all the rest just say "No Data"

In the "Web Usage", Threats", Predefined Reports" etc.  all of the charts just say "No Data"

it is running 4.0 MR3 patch 8 ( which is the last version for the 100B )

Does this sound familiar to anyone? Any help would be appreciated

Mark

[Carlos_A_Almeida]

Are you sure you did select monitor instead of allow at your web filter and application sensor security profiles?

[Mbutler522010]

it took me awhile to figure out what you were referring to!

I do show green checks (allow) for all things not blocked in the Fortiguard Categories in my Security Profile. I will try changing those to monitor...

Mark

[hzhao_FTNT]

It looks like utm logs are missing. What's your fortigate version? Can you see utm log on FAZ?

[Mbutler522010]

My fortigate is 5.2.0 GA

Strange...

I have entries in the "Event Log" and "Traffic Log" log listing when I select "All FortiGates" but do not see anything at all in the UTM log listing and I have "Show Consolidated UTM Log" checked in System > Admin > Settings"

[hzhao_FTNT]

From release note of FOS5.2.0

FortiOS v5.2.0 is supported by the following FortiManager and FortiAnalyzer software versions: • FortiManager v5.0.7 • FortiManager v5.2.0 • FortiAnalyzer v5.0.7 • FortiAnalyzer v5.2.0 You should upgrade the FortiManager and/or FortiAnalyzer prior to upgrading the FortiGate.

[Mbutler522010]

hmmm, unfortunately I have a FortiAnalyzer 100B and 4.0 MR3 patch 8 is the last release for it. I wonder if it is a compatibility problem.

Sadly - I turned on "monitor" for all categories on the fortigate (it had "allow") and this morning, the Analyzer shows I got another 3gb of logs,.....but I have the same trouble: "Traffic Summary for last 24 hours" shows no data.

Looks like I am going to have to call in a service request

Mark

[L_FTNT]

Could you run the following command from the CLI console to check the SQL database status?

diagnose sql status rebuild-db

when upgrade from 4.x to 5.x, it requires a SQL db rebuilding. The time it takes to rebuild can be vary depending on the amount of data you have in the database: it can be from a couple hours to a few days.

During this rebuilding stage, new logs from FGT will be received and saved on the disk but won't be inserted into the database until the rebuilding is completed. So you won't see the new logs from the log view and the reporting on the new logs won't have data in it.

[Mbutler522010]

This is the kind of thing that makes working on the Fortianalyzer so difficult. According to http://kb.fortinet.com/kb/documentLink.do?externalID=FD35225

Prior to FortiAnalyzer 5.2.1, the only direct method of determining the status of the rebuild is to use the following command: diagnose sql status rebuild-db

However, when I execute that command on my "FortiAnalyzer-100B v4.0,build0719 (MR3 Patch 8) " system it fails with a "not a valid command" error. specifically:

Connected
  
FortiAnalyzer-100B # ?
 config config object
 diagnose diagnose facility
 execute execute static commands
 exit exit CLI
 get get configuration
 show retrieve value
 
FortiAnalyzer-100B # diagnose sql status rebuild-db
 
command parse error before 'rebuild-db'
Input not as expected.
 
FortiAnalyzer-100B # diagnose sql status ?
 run_sql_rpt Show run_sql_rpt status.
 sqlplugind Show sqlplugind status.
 sqlreportd Show sqlreportd status.
 
FortiAnalyzer-100B # diagnose sql status 

FortiAnalyzer-100B #

None of the manuals seem to match what I can type at the CLI and very little info is shown. but I was able to get a debug report of the config and it shows some line stating an index needs to be updated...

ortiAnalyzer-100B # diag debug report

SYSTEM:

### get system status

Version: FortiAnalyzer-100B v4.0,build0719,131126 (MR3 Patch 8)
Branch point: 719
Release Version Information: MR3 Patch 8
Serial-Number: FL100B3107003610
BIOS version: 04000005
VCM Plugin Version: 1.217
Admin Domain Status: disabled
Max number of administrative domains: 1
Registered Devices: 2
Maximum Supported Devices: 100
Hostname: FortiAnalyzer-100B
FIPS mode: disabled
System Time: Wed Jan 14 14:02:15 PST 2015

Disk Usage: Free 177.11GB, Total 228.74GB
### get system performance

CPU states: 9% used, 5% used(Excluded NICE), 91% idle
CPU Usage:%user %nice %sys %idle %iowait%irq %softirq
4.45 4.59 4.01 86.30 0.61 0.00 0.05 
Memory states: 38% used
Uptime: 7 days, 5 hours, 14 minutes

### diagnose sys cpu_mem

CPU usage: 9%
cpu_num: 1.
CPU[0] usage: 13%
Memory usage: 38%

### diagnose fortiguard status

### diagnose report status

0 reports have been generated successfully, details:
started total: 0 scheduled: 0 manually: 0
finished successed: 0 killed: 0 failed: 0
process running: 0 wait: 0

Network/VPN:

### diagnose netlink device list

Inter-| Receive | Transmit | Link
 face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed | up
    lo: 152582223 614623 0 0 0 0 0 0 152582223 614623 0 0 0 0 0 0 -
 port4: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
 port3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
 port2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
 port1: 1509446990 2415121 0 0 0 0 0 0 268575438 1588315 0 0 0 0 0 0 1
 tunl0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -
  gre0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -
  sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -
ip6tnl0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -
tun_fgfm: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -

### diagnose vpn tunnel list

Devices/Disk usage:

### diagnose device status

Type Total Status
FortiGate 1 Device additions permitted
FortiManager 0 Device additions permitted
Syslog 0 Device additions permitted
FortiClient 0 Device additions permitted
FortiMail 1 Device additions permitted
FortiWeb 0 Device additions permitted
FortiCache 0 Device additions permitted

### diagnose sys sysinfo diskused

Total space: 228.74GB
Free space: 177.11GB
Space used: 22.57%

### diagnose sys diskusage

Local clients:
Local logs: 1000/4 MB
Network Analyzer: 1000/17302 MB

Registered clients:
FE-1003109001641 (FE-1003109001641): 40000/24378 MB (17586/6792)
FortiGate-HA_FG800C3912800902 (FG800C3912800902): 50000/19415 MB (12190/7225/0/0/0)

Unregistered clients:
SYSLOG-7F000001: 0/400 MB

Total client disk usage: 61099 MB

### diagnose log device

Device Name Device ID Used Space(logs/DLP/quar/IPS) Allocated Space % Used 
FE-1003109001641 FE-1003109001641 24378M(24378/ 0/ 0/ 0) 40000M 60.95%
FortiGate-HA_FG800C3912800902 FG800C3912800902 19415M(19415/ 0/ 0/ 0) 50000M 38.83%

RAID/Disks/File-system:
### diagnose sys disk health

Disk 1:
smartctl 5.39 2009-09-22 r2922 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-9 by Bruce Allen, http://smartmontools.sourceforge.net

SMART overall-health self-assessment test result: PASSED


### diagnose sys disk errors

Disk 1:
smartctl 5.39 2009-09-22 r2922 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-9 by Bruce Allen, http://smartmontools.sourceforge.net

SMART Error Log Version: 1
No Errors Logged

### diagnose sys fsystem

Log disk partition table type is MSDOS.
Log disk is ext3 file system.
Log disk directories are indexed.
Log disk has extended attributes enabled.

### diagnose sys file-system fsreport

No check results available.

Crash-Logs:

### diagnose debug crashlog list

httpd:
  core: 21360640 bytes, Thu Jan 8 16:09:13 PST 2015

Messages:

### diagnose fortilogd status

fortilogd is starting
config socket OK
cmdb socket OK
cmdb register log.device OK
cmdb register log.unregistered OK
cmdb register log.settings OK
cmdb register log.forwarding OK
cmdb register system.operation OK
log socket OK

### diagnose fortilogd msgrate

msgs/sec: 0.0, msgs/30sec: 2.3, msgs/60sec: 3.7

### diagnose fortilogd msgstat

Indexer:

### diagnose log-indexer status

5) Scan 0:
Checking /Storage/Logs/FE-1003109001641/slog.1413236910.log
Index needs updating.
Report binary file needs updating.
operation took: 0 s
1420653245( 7/Jan/15 09:54:05) Scan 0:
Checking /Storage/Logs/FE-1003109001641/slog.1414132022.log
Index needs updating.
Report binary file needs updating.
operation took: 0 s
1420653245( 7/Jan/15 09:54:05) Scan 0:
Checking /Storage/Logs/FE-1003109001641/slog.1415216360.log
Index needs updating.
Report binary file needs updating.
operation took: 0 s
1420653245( 7/Jan/15 09:54:05) Scan 0:
Checking /Storage/Logs/FE-1003109001641/slog.1416276809.log
Index needs updating.
Report binary file needs updating.
operation took: 0 s
1420653245( 7/Jan/15 09:54:05) Scan 0:
Checking /Storage/Logs/FE-1003109001641/slog.1417387275.log
Index needs updating.
Report binary file needs updating.
operation took: 0 s
1420653245( 7/Jan/15 09:54:05) Scan 0:
Checking /Storage/Logs/FE-1003109001641/slog.1418329897.log
Index needs updating.
Report binary file needs updating.
operation took: 0 s
1420653245( 7/Jan/15 09:54:05) Scan 1: Compute work done ...
compute work: index 1 / 173
compute work: bin 0 / 172
Compute Work Done: 1 seconds
1420653245( 7/Jan/15 09:54:05) Scan 1: Process 30 day files ...
1420653245( 7/Jan/15 09:54:05) Scan 1: Process active files ...
1420653245( 7/Jan/15 09:54:05) Scan 1:
Checking /Storage/Logs/.self/elog.log
Creating index ...
Creating binary file ...
operation took: 1 s
Mem: 12996 K, total 516324 K (2.52%)
1420653246( 7/Jan/15 09:54:06) Scan 1:
Checking /Storage/Logs/.self/nlog.log
/Storage/Logs/.self/nlog.log is older than the current scan period.
/Storage/Logs/.self/nlog.log has 1 strike.
1420653246( 7/Jan/15 09:54:06) Scan 1:
Checking /Storage/Logs/FE-1003109001641/elog.log
Creating index ...
Creating binary file ...
operation took: 22 s
Mem: 12996 K, total 516324 K (2.52%)
1420653268( 7/Jan/15 09:54:28) Scan 1:
Checking /Storage/Logs/FE-1003109001641/hlog.log
Creating index ...
Creating binary file ...
operation took: 80 s

### diagnose log-indexer bincheck


[Archived report binary status] Total: 172, Complete: 0

Of course I cannot find anything that tells me how to rebuild them. I tried a couple of different things but everything fails:

FortiAnalyzer-100B # diag log-indexer ?
 badlogs Show any logs that cannot be indexed.
 bincheck Check DB binary file status.
 error-msg Error messages.
 rebuild-db Rebuild the report binary files.
 recheck Flush the cache and recheck all logs.
 reindex-all Redo all the device index.
 reindex-custom Rebuild only indexes lacking the current custom log fields.
 reindex-device Redo one device index or one log type.
 status Running status.

FortiAnalyzer-100B # diag log-indexer reindex-all

FortiAnalyzer-100B # diag log-indexer rebuild-db
Warning! Do not run this unless you have been instructed to by support!
No device ID was specified, so all report binary files will be deleted
and it may take a significant amount of time to rebuild them.
Do you want to continue? (y/n)y

Failed to stop log_indexer.
Internal error.

FortiAnalyzer-100B # diag log-indexer badlogs


Logs that cannot be indexed: 0.


FortiAnalyzer-100B # execute log-integrity FE slog.1416276809.log
No validation action is configured.

FortiAnalyzer-100B #

..sigh..

[Dave_Hall]

Thing is about some/most of those commands is the context that they can be executed in -- if you have ADOMS configured, you need to switch to gobal before those commands will work.  may be try...

config global

diagnose sql status rebuild-db