Hi,
Everything looks fine except the FLOW CPU is spiking randomly, It mainly spikes because of the traffic. The session creation looks normal so there are chances that either ALG, bust in the same session or some other traffic invoking the high flow CPU. You may need to run the profiling on the device to see what traffic is causing the high CPU.
https://kb.juniper.net/InfoCenter/index?page=content&id=KB9453&actp=METADATA
Thanks,
VIkas
Thanks for clarifying this.
Hello,
After enabling traffic shaping on a policy that is covering traffic for multiple subnets.
After awhile I got compalins that for part of the destination no traffic was passing. After further checks I noticed that traffic that was meant for VPN tunnles was not passing even thought for other destinations (non VPN Tunnles) was working.
Has anyone encountered this issue ? is a bug related to screenOS?
I can confirm happening on a SSG520 and SSG140
So, I have put myself in a pickel. Here's my predicament and environment, and I'm sure one of you guru's can point me in the right direction as I think I exhausted my search and ways around this.
My initial goal was to update my backup/spare SSG20 and put an update config on it since we are moving to a new carrier soon and was going to test the config on the new circuit prior to cut. The backup was running 6.1.x and the production was at 6.3.17. I started with trying to update firmware to what I had on Production 6.3.17 and using firmware that I had from 2014. I first ran the update via Webgui and it failed, after connecting to console it was in a constant reboot, although I don't remember exactly what the error was I tried the update from TFTP and received the "invalid DSA signature, Bogus image....." error ... I then tried another downgraded version 6.3.14 (which I had from previous) which loaded and verified however continues to reboot and crash dump. I made the obvious assumption that any SW I download will require the updated key, so I went as far as saving the OS from production SSG and trying to load that. I also get invalid DSA error.
Below is the dump file....any assistance or next steps would be appreciated. I do NOT have a version of original OS before trying to upgrade(shame on me).
Juniper Networks SSG20 Boot Loader Version 1.3.2 (Checksum: A1EAB858)
Copyright (c) 1997-2006 Juniper Networks, Inc.
Total physical memory: 256MB
Test - Pass
Initialization - Done
Hit any key to run loader
Hit any key to run loader
Hit any key to run loader
Hit any key to run loader
Loading default system image from on-board flash disk...
Done! (size = 13,369,344 bytes)
Image authenticated!
Juniper Networks, Inc
SSG5/SSG20 System Software
Copyright, 1997-2008
Version 6.3.0r14.0
Cksum:b5127182
Load Manufacture Information ... Done
Initialize FBTL 0........ Done
Load NVRAM Information ... (6.3.0)Done
Install module init vectors
Changed to l3 mode
ixQMgrInit: IxQMgr already initialised
Install modules (01274800,020b8000) ...
PPP IP-POOL initiated, 256 pools
Initializing DI 1.1.0-ns
System config (17011 bytes) loaded
Done.
Load System Configuration ................................................................
Unsupported command - set zone "VLAN" block
........................................................................................................................modem is not detected
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Disabled licensekey auto update
..............................................Done
system init done..
login: Juniper Networks SSG20 Boot Loader Version 1.3.2 (Checksum: A1EAethernet0/4 interface change physical state to Up
bgroup0 interface change physical state to Up
System change state to Active(1)
###Crash Time: 17Oct2017:09:20:26###
System Level:
Image In Task Level
Current Task Is:sys up id = 88
*********************************************************
Exception Dump
*********************************************************
System up time: 0 hours 0 minutes 13 seconds
Version 6.3.0r14.0
Exception(Data Abort Exception code(1002))
Exception address: 001a2568
Registers of Main Processor:
R0: 00000000 R1: 00000001 R2: 00000093
R3: 01da9234 R4: 84b87508 R5: 89188294
R6: 7b478b30 R7: 03ad0fe0 R8: 00000024
R9: 00000000 R10(sl): 8bffff80 R11(fp): 8bfffee8
R12(ip): 7b478b0c R13(sp): 8bfffec8 r14: 001a2570
lr: 00562938 SPSR: 20000010 CPSR: 20000097
The registers of control processor 15:
CR1ARM: 000031FF CR1XSCALE: 00000000 CR2: 0f9cc000
CR3: 000000E7 CR4: Reserved CR5: 000000f5
CR6: 891882B4 CR7: N/A CR8: N/A
CR9: 00000000 CR10: N/A CR11: Reserve
CR12: Reserve CR13: 00000000
Stack dump:
8bfffe48: 03 ad 0f e0 00 00 00 24 00 00 00 00 8b ff ff 80
8bfffe58: 8b ff fe 88 8b ff fe 68 00 56 3b 70 00 ec f1 18
8bfffe68: 00 83 2a b0 00 00 00 00 00 00 00 00 84 b8 75 08
8bfffe78: 04 60 0d 8c 8b ff fe c4 8b ff fe 9c 00 1a 25 1c
8bfffe88: 00 56 3a f8 01 2c 44 b0 03 ad 0f e0 00 00 02 65
8bfffe98: 01 da 92 34 00 00 00 00 84 b8 75 08 04 60 0d 8c
8bfffea8: 00 00 00 38 03 ad 0f e0 00 00 00 24 8b ff ff 80
8bfffeb8: 8b ff fe e8 8b ff fe c8 00 1a 25 e0 00 1a 23 c4
8bfffec8: 00 00 00 64 02 ab 2d a0 04 60 0d 60 00 00 00 00
8bfffed8: 02 0e a3 68 8b ff fe f8 8b ff fe ec 00 1a 26 58
8bfffee8: 00 1a 25 50 8b ff ff 14 8b ff fe fc 00 1a 26 e8
8bfffef8: 00 1a 26 10 00 00 00 01 00 00 00 02 00 00 00 01
8bffff08: 8b ff ff 30 8b ff ff 18 00 1a ae 2c 00 1a 26 78
8bffff18: 8b ff ff 80 00 1a ac bc 00 00 00 00 8b ff ff 4c
8bffff28: 8b ff ff 34 00 82 6d ac 00 1a ac c8 00 00 00 04
8bffff38: 02 71 6f d8 02 e2 22 6c 8b ff ff 7c 8b ff ff 50
8bffff48: 00 82 6f 14 00 82 6d 2c 00 00 00 01 00 00 00 01
8bffff58: 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00
8bffff68: 00 00 00 00 00 00 00 00 8b ff ff ac 8b ff ff 80
8bffff78: 00 82 75 d8 00 82 6e 74 00 00 00 02 00 00 00 02
8bffff88: 00 00 00 01 8b ff ff bc 8b ff ff 9c 00 c5 6c 84
8bffff98: 00 08 1d 14 00 82 74 30 8b ff ff bc 8b ff ff b0
8bffffa8: 00 82 74 4c 00 82 75 50 00 00 00 00 8b ff ff c0
8bffffb8: 00 08 1d 14 00 82 74 3c 00 00 00 00
Trace Dump:
001a2568 00562938 001a2658 001a26e8 001aae2c 00826dac 00826f14 008275d8
0082744c 00081d14
FP Trace Dump:
00000000 00000000 8bfffee8 8bfffef8 8bffff14 8bffff30 8bffff4c 8bffff7c
8bffffac 8bffffbc
Crash dump, the system will reboot...
-----------
OS Context:
-----------
Died Flow/bootup Module
Cur Task Context: sys up
Crash dump is done.
sys up far = 89
did you ever get this fixed? what was your fix? I have similar issue with Spare?
also....the production SSG20 shows a key of all zero's, meaning that there is not a key installed. I suspect same for backup/spare.
Do you think it could be due to the Netscreen 50 being an old firewall and only having 100mb port capability? It connects to a gigabit switch but obviously only operates up to 100mb. Port utilization on the switch is showing 50%+ whereas all other ports are below 1% utilisation. Do you think the issue could be resolved by using a more modern firewall that supports gigabit speeds? It could just be struggling to handle the volume of traffic.
do you still happen to have you old version of screenos? I am in pickle and need to roll back to a version not on current download page, since it requires a diff key signature
This is possible depending on how you have your traffic shaping configured. You need to specify the interface bandwidth so the device knows the total bandwidth allowed. Also, when you enable traffic shaping on a policy, all traffic has to pass through the traffic shaping queue, not just the traffic for that policy. Any traffic that does not match a policy that has traffic shaping will be set to lowest priority.
This is a known issue when upgrading from 6.2r4 and below. It is caused by a change in the DHCP version.
If you are getting this message:
Error: Bogus image – not authenticated!!!
This error will occur if you upgrade to the new ScreenOS image and still have the OLD signing key on your device. The boot screen on the console port will show this message:
********Invalid image!!!
********Bogus image – not authenticated!!!
Fips check failed
Done
To recover from this error and allow the device to boot you need to delete the signing key.
delete crypto auth-key
Then reboot the device and the new ScreenOS should load.
loaded the hotfix from above - ssg5ssg20.6.2.0r4-ef2 got the same validation error. It looks like my spare has the old signature key. I was able to find 2 of my ssg5's from branches and viewed the key and extracted the firmware off via tftp. I didn't think they would work and they didn't. They were revision 6.3.0r8 and r14 so I was in same situation with constant loop boot error.
It looks like I need a pre 6.3 image to be able to boot again and then be able to change the key OR a way to change the key with TFTP.
I have a dropbox if anyone feels compelled to share an old firmware.
Having the same problem... droping packet only from the LAN to the WAN interface... but it reply just fine from the wan interface to the internet...
my internet circuit is 50 and we are barely using 20 ...
I have another site with even small ssg's without seeing this issue...
did you guys found a solution?
In my case I have 2 interfaces towards the internet which are both used for VPN's.
When I enable the traffic shaping on a policy it contains traffic for VPN's on both internet interfaces.
If you can please advise on the following:
How to determine which bandwidth is used for the shaping queue ?
How to determine when I get drops on a policy?
If there are any CLI commands that can come handy in troubleshooting this?
Thanks in advance,
George
If you are getting this error, you can try to delete the current key.
********Invalid image!!!
********Bogus image – not authenticated!!!
Fips check failed
Done
To recover from this error and allow the device to boot you need to delete the signing key.
delete crypto auth-key
Then reboot the device and the new ScreenOS should load.
was able to get the hotfix from Seibert with the old key. thx I have updated to 6.3.r17
last question... I assume now that I can get the unit to boot, I can delete the key signature. Based on reading the different forum and tech notes, the OS should still boot. My goal is to create same environment on both units, so I don't have this issue in future.
It's not necessary that you are hitting device limit. As per the data sheet it can handle up till 170Mbps total if it's clear text only and 45 Mbps with VPN. With total 118 concurrent sessions and 10 sessions/sec new sessions during the issue, it doesn't look to be too many devices in the network. There could be huge traffic through the single session, or some traffic pattern like ALG(SQL etc) etc could be invoking the high CPU. Also, please check if you have syslog configure on tcp or any broadcast, multicast etc coming to the device. You also can take a few seconds wireshark capture from your switch and what all are coming to the firewall and if everything is legitimate.
Thanks,
Vikas
The new signing keys were released some years ago. For the upgrades you simply download the new key and upload this first to the ScreenOS device. then perform the upgrade itself after this and all is well. The details are in these articles.
Signing Key Articles
http://kb.juniper.net/InfoCenter/index?page=content&id=TSB16495
http://kb.juniper.net/InfoCenter/index?page=content&id=TSB16496
Thanks guys. I have updated and back in business with my spare and ready for my cutover.
I've read why you would want the key signature, to make sure it is not corrupt or otherwise. However, from a functionality sense what does the key signature do for you. I haven't found anything specific pointing to that?
Thanks for the reply. I think we could potentially be hitting the 170mbps limit at times. The weird thing is it's fine for a week or so, then suddenly the latency jumps ridiculously high and stays like it until the firewall is rebooted. Then it's back to normal again for a week or so. Is there anything that could 'build up' on the firewall that takes it to breaking point? I'm convinced it's something in the firewall since when we reboot it it's all back to normal again. Any other ideas?