During our regular maintenance, after rebooted one SRX345, and found it stuck at db mode, which is debug mode.After a short and quick analysis, I found Juniper JunOS devices may get stuck in the boot process or fail to boot the OS, in rare cases, after a sudden power loss or ungraceful power shut down. Juniper routers, switches and firewalls can experience file system corruption, which prevents the device from recovering to a functional state. It is recommended that customers minimize their log configurations to prevent excessive read/writes to the file system, which reduces the stress on storage media and reduces the potential occurrence of this issue. Moreover, if abrupt power failures are transient for a very short period of time, the availability of an UPS can also prevent the device from experiencing a sudden power loss.
You do not have to worry about damaging hardware in these situations, as the hardware cannot tell the difference between a graceful shutdown and pulling the power cord. The potential for damage is with the file system structure. It is possible for data to be corrupted, when the computer's power is interrupted with the operating system running. The data could be in the nodes, which could result in files being lost or file contents being corrupted.
Although rare, this issue more likely occurs on platforms that use a UNIX/BSD-based operating system, such as Junos, to access the flash-based storage media.
“Although rare, file system damage can occur with an abrupt power off, which may cause problems on the next boot. Use the request system halt or request system reboot command to gracefully shut down or reboot the OS. Once the OS is halted, it is safe to remove power.” - from O'Reilly Media’s JUNOS Enterprise Switching book.
There are a couple of KB discussing the fix. KB29811 is using a USB to copy a snapshot from healthy device to faulty device. KB20046 suggest to press space to go to u-boot prompt and enter some commands to fix issue.
db> help
DDB Quick Help
-------------------
Type 'c' to continue, 'reset' or 'panic' to restart.
print p examine x search set write
w delete d break dwatch watch dhwatch
hwatch step s continue c until next
match trace alltrace where bt call show
ps gdb reset kill watchdog thread panic
ddbdumpsys dumpsys
db> reset
The first reset brought me into RAM mode.
SPI stage 1 bootloader (Build time: May 3 2016 - 23:48:30) early_board_init: Board type: SRX_345 U-Boot 2013.07-JNPR-3.1 (Build time: May 03 2016 - 23:48:31) SRX_345 board revision major:1, minor:7, serial #: CZ2616AF0301 OCTEON CN7130-AAP pass 1.2, Core clock: 1600 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x10fc00000, size: 0x400000 DRAM: 4 GiB Clearing DRAM...... done Using default environment SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB Found valid SPI bootloader at offset: 0x90000, size: 1481840 bytes U-Boot 2013.07-JNPR-3.1 (Build time: May 03 2016 - 23:50:19) Using DRAM size from environment: 4096 MBytes checkboard siege SATA0: not available SATA1: not available SATA BIST STATUS = 0x0 SRX_345 board revision major:1, minor:7, serial #: CZ2616AF0301 OCTEON CN7130-AAP pass 1.2, Core clock: 1600 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x10f000000, size: 0x1000000 DRAM: 4 GiB Clearing DRAM...... done SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB PCIe: Port 0 link active, 1 lanes, speed gen2 PCIe: Link timeout on port 1, probably the slot is empty PCIe: Port 2 not in PCIe mode, skipping Net: octrgmii0 octeon_fdt_broadcom_config: Unknown broadcom phy for octrgmii0 Interface 4 has 1 ports (AGL) Type the command 'usb start' to scan for USB storage devices. Boot Media: eUSB usb Found TPM SLB9660 TT 1.2 by Infineon TPM initialized Hit any key to stop autoboot: 0 Octeon srx_345_ram#
Octeon srx_345_ram# help ? - alias for 'help' base - print or set address offset base64 - print or set address offset boot - boot default, i.e., run 'bootcmd' bootd - boot default, i.e., run 'bootcmd' bootelf - Boot from an ELF image in memory bootloader - bootloader - upgrade u-boot bootloader - upgrade loader bootloader - upgrade ushell bootloader - check u-boot bootloader - check loader bootm - boot application image from memory bootoct - Boot from an Octeon Executive ELF image in memory bootoctelf - Boot a generic ELF image in memory. NOTE: This command does not support simple executive applications, use bootoct for those. bootoctlinux- Boot from a linux ELF image in memory bootp - boot image via network using BOOTP/TFTP protocol bootvx - Boot vxWorks from an ELF image cdp - Perform CDP network configuration cmp - memory compare cmp64 - memory compare cp - memory copy cp64 - memory copy cpld - cpld - peek/poke CPLD crc32 - checksum calculation dhcp - boot image via network using DHCP/TFTP protocol dns - lookup the IP of a hostname echo - echo args to console eeprom - EEPROM sub-system env - environment handling commands fanspd - fan speed - set fan speed fatinfo - print information about filesystem fatload - load binary file from a dos filesystem fatls - list files in a directory (default /) fatwrite - write file into a dos filesystem fdt - flattened device tree utility commands flush_dcache- Flushes and invalidates the data cache flush_l2c - Flushes the L2 cache freeprint - Print list of free bootmem blocks go - start application at address 'addr' gpio - input/set/clear/toggle gpio pins help - print command description/usage i2c - I2C sub-system id_eeprom - id_eeprom - peek/poke EEPROM inv_icache - Invalidates the instruction cache jump_mdk_mem- jump_mdk_mem - jump to mdk entry loadb - load binary file over serial line (kermit mode) loads - load S-Record file over serial line loady - load binary file over serial line (ymodem mode) loop - infinite loop on address range loop64 - infinite loop on address range md - memory display md5sum - compute MD5 message digest md64 - memory display mdio - MDIO utility commands mdkinit - mdkinit - start MDK meminfo - display memory information mii - MII utility commands mm - memory modify (auto-incrementing address) mm64 - memory modify (auto-incrementing address) mw - memory write (fill) mw64 - memory write (fill) namedalloc - Allocate a named bootmem block namedfree - Free a named bootmem block namedprint - Print list of named bootmem blocks nfs - boot image via network using NFS protocol nm - memory modify (constant address) nm64 - memory modify (constant address) octreginfo - print register information octwd - Starts the OCTEON watchdog pca953x - pca953x gpio access pci - list and access PCI Configuration Space ping - send ICMP ECHO_REQUEST to network host printenv - print environment variables qlm - Octeon QLM debug function (dangerous - remove from final product) read64 - read 64 bit word from 64 bit address (deprecated) read64b - read 8 bit word from 64 bit address (deprecated) read64l - read 32 bit word from 64 bit address (deprecated) read64s - read 16 bit word from 64 bit address (deprecated) reset - Perform RESET of the CPU run - run commands in an environment variable saveenv - save environment variables to persistent storage saves - save S-Record file over serial line setenv - set environment variables sf - SPI flash sub-system sha1sum - compute SHA1 message digest sleep - delay execution for some time sspi - SPI utility command test_mdk_mem- test_mdk_mem - start mdk at address 'addr' tftp - alias for tftpboot command (deprecated) tftpboot - boot image via network using TFTP protocol tftpput - TFTP put command, for uploading files to a server tftpsrv - act as a TFTP server and boot the first received file tlv_eeprom - EEPROM data parsing tpm - Issue a TPM command tpm_test - tpm_test - Test TPM usb - USB sub-system usbboot - boot from USB device version - print monitor, compiler and linker version write64 - write 64 bit word to 64 bit address (deprecated) write64b - write 8 bit word to 64 bit address (deprecated) write64l - write 32 bit word to 64 bit address (deprecated) write64s - write 16 bit word to 64 bit address (deprecated) Octeon srx_345_ram#
I tried a couple of time reset , reboot, and finally system load backup image.
loader> reboot Resetting... SPI stage 1 bootloader (Build time: May 3 2016 - 23:48:30) early_board_init: Board type: SRX_345 U-Boot 2013.07-JNPR-3.1 (Build time: May 03 2016 - 23:48:31) SRX_345 board revision major:1, minor:7, serial #: CZ2616AF0301 OCTEON CN7130-AAP pass 1.2, Core clock: 1600 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x10fc00000, size: 0x400000 DRAM: 4 GiB Clearing DRAM...... done Using default environment SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB Found valid SPI bootloader at offset: 0x90000, size: 1481840 bytes U-Boot 2013.07-JNPR-3.1 (Build time: May 03 2016 - 23:50:19) Using DRAM size from environment: 4096 MBytes checkboard siege SATA0: not available SATA1: not available SATA BIST STATUS = 0x0 SRX_345 board revision major:1, minor:7, serial #: CZ2616AF0301 OCTEON CN7130-AAP pass 1.2, Core clock: 1600 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x10f000000, size: 0x1000000 DRAM: 4 GiB Clearing DRAM...... done SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB PCIe: Port 0 link active, 1 lanes, speed gen2 PCIe: Link timeout on port 1, probably the slot is empty PCIe: Port 2 not in PCIe mode, skipping Net: octrgmii0 octeon_fdt_broadcom_config: Unknown broadcom phy for octrgmii0 Interface 4 has 1 ports (AGL) Type the command 'usb start' to scan for USB storage devices. Boot Media: eUSB usb Found TPM SLB9660 TT 1.2 by Infineon TPM initialized Hit any key to stop autoboot: 0 SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB SF: 1048576 bytes @ 0x200000 Read: OK ## Starting application at 0x8f0000a0 ... Consoles: U-Boot console Found compatible API, ver. 3.1 USB1: Starting the controller USB XHCI 1.00 scanning bus 1 for devices... 2 USB Device(s) found USB0: Starting the controller USB XHCI 1.00 scanning bus 0 for devices... 1 USB Device(s) found scanning usb for storage devices... 1 Storage Device(s) found FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.8 (slt-builder@svl-ssd-build-vm06.juniper.net, Tue Feb 10 00:32:30 PST 2015) Memory: 4096MB SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB [2]Booting from eUSB slice 1 Loading /boot/defaults/loader.conf /kernel data=0xb7fc40+0x1515a4 syms=[0x4+0x9dd10+0x4+0xeb2b7] Hit [Enter] to boot immediately, or space bar for command prompt. Booting [/kernel]... Kernel entry at 0x801000c0 ... init regular console Primary ICache: Sets 16 Size 128 Asso 39 Primary DCache: Sets 8 Size 128 Asso 32 Secondary DCache: Sets 1024 Size 128 Asso 4 CIU_FUSE 0xf/0xf GDB: debug ports: uart GDB: current port: uart KDB: debugger backends: ddb gdb KDB: current backend: ddb kld_map_v: 0x8ff80000, kld_map_p: 0x0 Running in PARTITIONED TLB MODE Copyright (c) 1996-2016, Juniper Networks, Inc. All rights reserved. Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. JUNOS 15.1X49-D45 #0: 2016-04-25 07:29:58 UTC builder@ralenth.juniper.net:/volume/build/junos/15.1/service/15.1X49-D45/obj/octeon/junos/bsd/kernels/JSRXNLE/kernel can't re-use a leaf (debug)! JUNOS 15.1X49-D45 #0: 2016-04-25 07:29:58 UTC builder@ralenth.juniper.net:/volume/build/junos/15.1/service/15.1X49-D45/obj/octeon/junos/bsd/kernels/JSRXNLE/kernel real memory = 4294967296 (4194304K bytes) avail memory = 2621882368 (2500MB) FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot) Security policy loaded: Junos MAC/veriexec (mac_veriexec) Security policy loaded: JUNOS MAC/pcap (mac_pcap) MAC/veriexec fingerprint module loaded: SHA256 MAC/veriexec fingerprint module loaded: SHA1 netisr_init: !debug_mpsafenet, forcing maxthreads from 4 to 1 cpu0 on motherboard : CAVIUM's OCTEON 70XX/71XX CPU Rev. 0.2 with no FPU implemented L1 Cache: I size 78kb(128 line), D size 32kb(128 line), thirty two way. L2 Cache: Size 512kb, 4 way obio0 on motherboard uart0: <Octeon-16550 channel 0> on obio0 uart0: console (9600,n,8,1) twsi0 on obio0 set clock 0x58 xhci0: <Cavium Octeon 7xxx xHCI Host Driver> on obio0 usb0: <USB bus for xHCI Controller> on xhci0 usb0: USB revision 3.0 uhub0: vendor 0x0000 XHCI root hub, class 9/0, rev 3.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered xhci1: <Cavium Octeon 7xxx xHCI Host Driver> on obio0 usb1: <USB bus for xHCI Controller> on xhci1 usb1: USB revision 3.0 uhub1: vendor 0x0000 XHCI root hub, class 9/0, rev 3.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered cpld0 on obio0 pcib0: <Cavium on-chip PCIe HOST bridge> on obio0 Disabling Octeon big bar support pcib0: Initialized controller pci0: <PCI bus> on pcib0 pci0: <network, ethernet> at device 0.0 (no driver attached) pci0: <network, ethernet> at device 0.1 (no driver attached) gblmem0 on obio0 octpkt0: <Octeon RGMII> on obio0 cfi0: <Macronix MX25L64 - 8MB> on obio0 cfi1: <Macronix MX25L64 - 8MB> on obio0 octagl0: <Octeon AGL> on obio0 umass0: ATP Electronics ATP CG eUSB, rev 2.00/11.00, addr 2 miibus0: <MII bus> on octagl0 brgphy0: <BCM54616S 10/100/1000baseTX PHY> on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto Timecounter "mips" frequency 1600000000 Hz quality 0 Registered AMT tunnel Encap with UDP Tunnel! Loading Redundant LT driver ###PCB Group initialized for udppcbgroup ###PCB Group initialized for tcppcbgroup Kernel thread "wkupdaemon" (pid 48) exited prematurely. da0 at umass-sim0 bus 0 target 0 lun 0 da0: <ATP ATP CG eUSB 1100> Fixed Direct Access SCSI-4 device da0: 40.000MB/s transfers da0: 7672MB (15712256 512 byte sectors: 255H 63S/T 978C) Trying to mount root from ufs:/dev/da0s1a MFSINIT: Initialising MFSROOT Process-1 beginning MFSROOT initialization... Creating MFSROOT... /dev/md0: 20.0MB (40956 sectors) block size 16384, fragment size 2048 using 4 cylinder groups of 5.00MB, 320 blks, 640 inodes. super-block backups (for fsck -b #) at: 32, 10272, 20512, 30752 Populating MFSROOT... Creating symlinks... Setting up mounts... Continuing boot from MFSROOT... Attaching /cf/packages/junos via /dev/mdctl... Mounted junos package on /dev/md1... O WARNING: R/W mount of /cf/var denied. Filesystem is not clean - run fsck mount: /dev/bo0s3f : Operation not permitted chflags: /var/packages/*: No such file or directory umount: /dev/bo0s3f: unknown file system Automatic reboot in progress... ** /dev/da0s1a (NO WRITE) ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 163 files, 115537 used, 1148817 free (65 frags, 143594 blocks, 0.0% fragmentation) mount reload of '/' failed: Operation not supported Verified junos signed by PackageProductionEc_2016 Verified jboot signed by PackageProductionEc_2016 Verified junos-15.1X49-D45-domestic signed by PackageProductionEc_2016 Checking integrity of BSD labels: s1: Passed s2: Passed s3: Passed s4: Passed ** /dev/bo0s3e ** Last Mounted on /config ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 18 files, 65 used, 94711 free (39 frags, 11834 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ** /dev/bo0s3f ** Last Mounted on /cf/var ** Phase 1 - Check Blocks and Sizes INCORRECT BLOCK COUNT I=70677 (480 should be 416) CORRECT? yes ** Phase 2 - Check Pathnames DIRECTORY CORRUPTED I=117761 OWNER=0 MODE=40755 SIZE=512 MTIME=Sep 18 14:41 2017 DIR=? SALVAGE? yes MISSING '.' I=117761 OWNER=0 MODE=40755 SIZE=512 MTIME=Sep 18 14:41 2017 DIR=? FIX? yes MISSING '..' I=117761 OWNER=0 MODE=40755 SIZE=512 MTIME=Sep 18 14:41 2017 DIR=/etc FIX? yes ** Phase 3 - Check Connectivity UNREF DIR I=117766 OWNER=0 MODE=40755 SIZE=512 MTIME=Sep 28 06:06 2016 RECONNECT? yes DIR I=117766 CONNECTED. PARENT WAS I=117761 UNREF DIR I=117762 OWNER=0 MODE=40755 SIZE=512 MTIME=Oct 17 21:07 2016 RECONNECT? yes DIR I=117762 CONNECTED. PARENT WAS I=117761 ** Phase 4 - Check Reference Counts UNREF FILE I=47130 OWNER=0 MODE=100644 SIZE=0 MTIME=Sep 18 14:40 2017 RECONNECT? yes LINK COUNT DIR I=117761 OWNER=0 MODE=40755 SIZE=512 MTIME=Sep 18 14:41 2017 COUNT 4 SHOULD BE 2 ADJUST? yes UNREF FILE I=117765 OWNER=0 MODE=120755 SIZE=22 MTIME=Sep 28 06:06 2016 RECONNECT? yes UNREF FILE I=117767 OWNER=0 MODE=100644 SIZE=1817 MTIME=Aug 21 01:10 2017 RECONNECT? yes UNREF FILE I=117794 OWNER=0 MODE=100644 SIZE=40960 MTIME=Sep 18 14:41 2017 RECONNECT? yes LINK COUNT FILE I=117794 OWNER=0 MODE=100644 SIZE=40960 MTIME=Sep 18 14:41 2017 COUNT 2 SHOULD BE 1 ADJUST? yes UNREF FILE I=117797 OWNER=0 MODE=100600 SIZE=40960 MTIME=Sep 18 14:41 2017 RECONNECT? yes UNREF FILE I=117800 OWNER=0 MODE=100644 SIZE=1195 MTIME=Sep 18 14:41 2017 RECONNECT? yes UNREF FILE I=117801 OWNER=0 MODE=100644 SIZE=40960 MTIME=Aug 21 01:10 2017 RECONNECT? yes UNREF FILE I=117802 OWNER=0 MODE=100600 SIZE=40960 MTIME=Aug 21 01:10 2017 RECONNECT? yes UNREF FILE I=117803 OWNER=0 MODE=100644 SIZE=1195 MTIME=Aug 21 01:10 2017 RECONNECT? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 459 files, 277705 used, 831508 free (340 frags, 103896 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** Checking integrity of licenses: Checking integrity of configuration: rescue.conf.gz: Passed cd: can't cd to /etc/db/pkg hw.re.gres_sync_other: 0 -> 1 Loading configuration ... Time and ticks drifted too much, resetting synchronization... Non-existant dump device /dev/bo0s1b mgd: commit complete Setting initial options: . Starting optional daemons: usbd. Doing initial network setup:. Initial interface configuration: additional daemons: eventd. Non-existant dump device /dev/bo0s1b Additional routing options:kern.module_path: /boot//kernel;/boot/modules -> /boot/modules;/modules/ifpfe_drv;IPsec: Initialized Security Association Processing. /modules; k. Doing additional network setup:. Starting final network daemons:. setting ldconfig path: /usr/lib /opt/lib starting standard daemons: cron. Initial rc.mips initialization:. Local package initialization:. starting local daemons:hw.re.gres_sync_other: 1 -> 0 set cores for group access . kern.securelevel: -1 -> 1 Creating JAIL MFS partition... JAIL MFS partition created Boot media /dev/da0 has dual root support WARNING: JUNOS versions running on dual partitions are not same ** /dev/da0s2a FILE SYSTEM CLEAN; SKIPPING CHECKS clean, 1152347 free (59 frags, 144036 blocks, 0.0% fragmentation) Wed Oct 4 13:26:20 UTC 2017 OOOOOOOOOOOOOO fw-prod-1 (ttyu0) login: john Password: Login incorrect login: Sec3166 Password: Login incorrect login: login: john Password: --- JUNOS 15.1X49-D45 built 2016-04-25 07:29:58 UTC *********************************************************************** ** ** ** WARNING: THIS DEVICE HAS BOOTED FROM THE BACKUP JUNOS IMAGE ** ** ** ** It is possible that the primary copy of JUNOS failed to boot up ** ** properly, and so this device has booted from the backup copy. ** ** ** ** Please re-install JUNOS to recover the primary copy in case ** ** it has been corrupted and if auto-snapshot feature is not ** ** enabled. ** ** ** *********************************************************************** {secondary:node0} john@fw-prod-1> {secondary:node0} john@fw-prodtrans-mex1-1> request system reboot Reboot the system ? [yes,no] (no) yes Shutdown NOW! [pid 10707] *** FINAL System shutdown message from john@fw-prodtrans-mex1-1 *** System going down IMMEDIATELY {secondary:node0} john@fw-prodtrans-mex1-1> OOWaiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `vnlru_mem' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...0 0 0 done syncing disks... Syncing disks, buffers remaining... 10 10 10 10 10 8 8 8 8 8 8 8 4 4 4 4 4 4 4 2 2 2 2 2 2 2 Final sync complete Uptime: 6d7h9m4s Rebooting... cpu_reset: Stopping other CPUs SPI stage 1 bootloader (Build time: May 3 2016 - 23:48:30) early_board_init: Board type: SRX_345 U-Boot 2013.07-JNPR-3.1 (Build time: May 03 2016 - 23:48:31) SRX_345 board revision major:1, minor:7, serial #: CZ2616AF0301 OCTEON CN7130-AAP pass 1.2, Core clock: 1600 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x10fc00000, size: 0x400000 DRAM: 4 GiB Clearing DRAM...... done Using default environment SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB Found valid SPI bootloader at offset: 0x90000, size: 1481840 bytes U-Boot 2013.07-JNPR-3.1 (Build time: May 03 2016 - 23:50:19) Using DRAM size from environment: 4096 MBytes checkboard siege SATA0: not available SATA1: not available SATA BIST STATUS = 0x0 SRX_345 board revision major:1, minor:7, serial #: CZ2616AF0301 OCTEON CN7130-AAP pass 1.2, Core clock: 1600 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR) Base DRAM address used by u-boot: 0x10f000000, size: 0x1000000 DRAM: 4 GiB Clearing DRAM...... done SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB PCIe: Port 0 link active, 1 lanes, speed gen2 PCIe: Link timeout on port 1, probably the slot is empty PCIe: Port 2 not in PCIe mode, skipping Net: octrgmii0 octeon_fdt_broadcom_config: Unknown broadcom phy for octrgmii0 Interface 4 has 1 ports (AGL) Type the command 'usb start' to scan for USB storage devices. Boot Media: eUSB usb Found TPM SLB9660 TT 1.2 by Infineon TPM initialized Hit any key to stop autoboot: 0 SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB SF: 1048576 bytes @ 0x200000 Read: OK ## Starting application at 0x8f0000a0 ... Consoles: U-Boot console Found compatible API, ver. 3.1 USB1: Starting the controller USB XHCI 1.00 scanning bus 1 for devices... 2 USB Device(s) found USB0: Starting the controller USB XHCI 1.00 scanning bus 0 for devices... 1 USB Device(s) found scanning usb for storage devices... 1 Storage Device(s) found FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.8 (slt-builder@svl-ssd-build-vm06.juniper.net, Tue Feb 10 00:32:30 PST 2015) Memory: 4096MB SF: Detected MX25L6405D with page size 256 Bytes, erase size 64 KiB, total 8 MiB [0]Booting from eUSB slice 2 \ can't load '/kernel' can't load '/kernel.old' Press Enter to stop auto bootsequencing and to enter loader prompt. Type '?' for a list of commands, 'help' for more detailed help. loader> reboot Resetting... SPI stage 1 bootloader (Build time: May 3 2016 - 23:48:30) early_board_init: Board type: SRX_345
In KB26175, if the following error messages has been displayed , format install will be the last recovery option to resolve it.
can't load '/kernel'
can't load '/kernel.old'
The backup image is not same as primary. The fix is to install the required image 15.1x49-D50.3 to primary partition and reboot it.
john@fw-prodtrans-1> show version
node0:
--------------------------------------------------------------------------
Hostname: fw-prodtrans-1
Model: srx345
Junos: 15.1X49-D45
JUNOS Software Release [15.1X49-D45]
node1:
--------------------------------------------------------------------------
Hostname: fw-prodtrans-2
Model: srx345
Junos: 15.1X49-D50.3
JUNOS Software Release [15.1X49-D50.3]
john@fw-prodtrans-1# run show chassis routing-engine node 0 node0: -------------------------------------------------------------------------- Routing Engine status: Temperature 35 degrees C / 95 degrees F CPU temperature 70 degrees C / 158 degrees F Total memory 4096 MB Max 1229 MB used ( 30 percent) Control plane memory 2624 MB Max 551 MB used ( 21 percent) Data plane memory 1472 MB Max 662 MB used ( 45 percent) 5 sec CPU utilization: User 6 percent Background 0 percent Kernel 1 percent Interrupt 0 percent Idle 94 percent Model RE-SRX345 Serial ID CZ2616AF0301 Start time 2017-10-04 13:22:28 UTC Uptime 6 days, 7 hours, 5 minutes, 56 seconds Last reboot reason 0x800:reboot due to exception Load averages: 1 minute 5 minute 15 minute 0.05 0.08 0.03
{secondary:node0}
john@fw-prodtrans-1> show chassis cluster status
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring
CF Config Sync monitoring
Cluster ID: 9
Node Priority Status Preempt Manual Monitor-failures
Redundancy group: 0 , Failover count: 0
node0 0 secondary no no CF
node1 100 primary no no None
Redundancy group: 1 , Failover count: 0
node0 0 secondary no yes CF
node1 255 primary no yes None
{secondary:node0}
john@fw-prodtrans-1> show security flow session summary
node0:
--------------------------------------------------------------------------
Unicast-sessions: 128
Multicast-sessions: 0
Failed-sessions: 0
Sessions-in-use: 133
Valid sessions: 128
Pending sessions: 0
Invalidated sessions: 5
Sessions in other states: 0
Maximum-sessions: 384000
node1:
--------------------------------------------------------------------------
Unicast-sessions: 123
Multicast-sessions: 0
Failed-sessions: 0
Sessions-in-use: 139
Valid sessions: 123
Pending sessions: 0
Invalidated sessions: 16
Sessions in other states: 0
Maximum-sessions: 384000
{secondary:node0} john@fw-prodtrans-1> request system software add /var/tmp/junos-15.1X49-D50.3-domestic.tgz no-copy no-validate reboot
Oct 10 20:40:53 init: gstatd (PID 2300) exited with status=1
Oct 10 20:40:53 init: exec_command: /usr/sbin/gstatd (PID 2308) started
Oct 10 20:40:53 init: gstatd (PID 2308) started
Formatting alternate root (/dev/da0s2a)...
/dev/da0s2a: 2518.0MB (5156848 sectors) block size 16384, fragment size 2048
using 14 cylinder groups of 183.62MB, 11752 blks, 23552 inodes.
super-block backups (for fsck -b #) at:
32, 376096, 752160, 1128224, 1504288, 1880352, 2256416, 2632480, 3008544,
3384608, 3760672, 4136736, 4512800, 4888864
Installing package '/altroot/cf/packages/install-tmp/junos-15.1X49-D50.3-domestic' ...
Verified junos-boot-srxsme-15.1X49-D50.3.tgz signed by PackageProductionEc_2016
Verified junos-srxsme-15.1X49-D50.3-domestic signed by PackageProductionEc_2016
Verified junos-boot-srxsme-15.1X49-D50.3.tgz signed by PackageProductionRSA_2016
Verified junos-srxsme-15.1X49-D50.3-domestic signed by PackageProductionRSA_2016
JUNOS 15.1X49-D50.3 will become active at next reboot
cp: cannot overwrite directory /altroot/cf/etc/ssh with non-directory /cf/etc/ssh
Saving state for rollback ...
Rebooting ...
shutdown: [pid 2671]
Shutdown NOW!
*** FINAL System shutdown message from root@fw-prodtrans-1 ***
System going down IMMEDIATELY
After successfully installed new image , reboot was smooth to get me into regular login window.
No comments:
Post a Comment