it depends what you would like to troubleshoot. If you like to troubleshoot the Phase1/2 of a VPN your command is the way to go which means:
diag debug reset
diag debug disable
diag debug application ike -1
The -1 means all message of debug in Phase1/2 but there are more debug levels for specific information:
2 Shows config changes
4 Shows connections which will be established
8 Only Phase-1 as Phase-2 comunications messages
16 Shows only NAT-T (Nat-Traversal)
32 Shows only DPD
64 Shows only Encryption/Decryption Key's
128 Shows only Encryption Traffic payload
You can set as you already mentioned a filter for ike which means:
# diagnose vpn ike log-filter ?
clear Erase the current filter.
dst-addr4 IPv4 destination address range to filter by.
dst-addr6 IPv6 destination address range to filter by.
dst-port Destination port range to filter by.
interface Interface that IKE connection is negotiated over.
list Display the current filter.
name Phase1 name to filter by.
negate Negate the specified filter parameter.
src-addr4 IPv4 source address range to filter by.
src-addr6 IPv6 source address range to filter by.
src-port Source port range to filter by.
vd Index of virtual domain. -1 matches all.
Keep in mind that application ike only shows Phase1/2 traffic nothing else. If you like to too look to the particular traffic from a user you can use the sniffer command which you will run on the ipsec interface and also there you can use different filters like the source IP of the user (IP Pool address). The sniffer command is used in following way:
# diagnose sniffer packet <interface_name> <‘filter’> <verbose> <count> a <interface_name>
Name of the interface to run the sniffer like wan1 etc. you can use also any for all interfaces! <‘filter’>
Definition for filter. The filter must be defined within "quotes" but you can use ‘ ' ". If you define none no filter
is applied any everthing is shwon <count>
Definition of counter which means how many packets will be shown Syntax:
'[ [src|dst] host<host_name_or_IP1> ] [ [src|dst] host<host_name_or_IP2> ] [ [arp|ip|gre|esp|udp|tcp] [port_no] ] [ [arp|ip|gre|esp|udp|tcp] [port_no] ]'
Not Port 443 = '!port 443'
Port 443 = 'port 443'
Host = 'host 192.168.1.1'
Host und Host = 'host 192.168.1.1 and host 192.168.1.2'
Host und Port 443 = 'host 192.168.1.1 and port 443'
Host und nicht Port 443 = 'host 192.168.1.1 and !port 443'
Host oder Port 443 = 'host 192.168.1.1 or port 443'
Nur udp Traffic = 'udp'
Nur SYN Flag = 'tcp&2==2'
Nur ARP Packete = 'arp' <verbose>
Definiert den Level der "Verbosity":
1 - Shows the header of a packet
2 - Shows the header and data of IP packets
3 - Shows the header and data of Ethernet Packets (Frames ACSII und HEX)
4 - Shows the header and Interface Name of Packets
5 - Shows the header and data of IP Packets with Interface Name
6 - Shows the headerund and of Ethernet Packets with Interface Name
This information is from following KB: http://kb.fortinet.com/kb/documentLink.do?externalId=11186
Actually the sniffer command is working like a tcpdump on linux/unix systems.
I only use capturing if you need application based information this means if a application is not working. If the access itself is a problem like destination is not answering or interrupt like you will find out who is sending a reset use sniffer command. You can also deeply troubleshoot based on TCP headers like follwoing example which shows only RST packets:
# diagnose sniffer packet any "tcp & 4 != 0"
The "tcp & 4 != 0" is indicating RST. How to get there see attached file (needs deeply tcp knowledge)! Below some examples:
'tcp==1' Only packets with FIN bit value with 1
'tcp&4==4' All packets with RST bit value with 1
'tcp&8==8' All packets with PSH bit value with 1
'tcp==16' Only packets with ACK bit value with 1
'tcp&32==32' All packets with URG bit value with 1
'tcp&64==64' All packets with ECE bit value with 1
'tcp&128==128' All packets with CWR bit value with 1
'tcp==24' Only packets with PSH und ACK bit value with 1
Hope this helps