Network Management

 View Only

  • 1.  IMC Remote DB Backup Failure

    Posted Jul 05, 2023 12:05 PM

    I have started receiving notifications that our IMC database backups are failing. I can manually backup using the IMC monitoring agent tool, but scheduled backups fail. 

    in the dbman_debug log, I see the "access denied error 5 message. I think this is related to a permissions issue, but the SQL Server agent (remote db) has access to the folder the backups are written to. Here is the snippet of the backup from the debug log from the remote database server. 

    What am I missing?  I am runnning IMC version E0708P03.

    Any advice would be much appreciated.

    Thanks,

    Matt

    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [TimeBackup] Begin auto backup .
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [CommonBackup] Begin backup ...
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [Remove] Fail to remove file: C:/Program Files/iMC/dbman\etc\dbman_bak.conf
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [CommonBackup] Echo all server, Flag=0
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [Remove] Remove file: C:\Program Files\iMC\dbman\bin\..\..\dbman\etc\LastOperationResult.conf
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [CLorOper::modifyLor] Write file LastOperationResult.conf: Result=Backup in progress.
    ErrorReport=You cannot perform backup when another backup operation is in progress.
    Tips=If you cannot obtain the failure reason from the ErrorReport, view the dbman/log/dbman.log and dbman_debug.log files.
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [WriteFile] Write file C:\Program Files\iMC\dbman\bin\..\..\dbman\etc\LastOperationResult.conf.
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [CDbBackup::BackupAllDataBase] Begin to auto backup all databases
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [CDbBackup::BackupAllDataBase] Path: db.db.db.183@aclm_db_imc_aclm_db_20230705_040001_full.db
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [CDbBackup::BackupOneLocalDBase] Ready to backup 'C:\Backups\db.db.db.183@aclm_db_imc_aclm_db_20230705_040001_full.db'
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [CDbBase::ExecSql] Input sql: BACKUP DATABASE aclm_db
    TO DISK = 'C:\Backups\db.db.db.183@aclm_db_imc_aclm_db_20230705_040001_full.db'
    WITH INIT
    GO

    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [WriteFile] Write file C:\Program Files\iMC\dbman\bin\dbop.sql.
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [runCommand] runCommand result FAIL. ret=1, errno=2, WIFEXITED(ret)=1, WEXITSTATUS(ret)=1
    2023-07-05 04:00:01 [THREAD(6296)] [ERROR] [CDbBase::ExecSql] Exec command fail
    2023-07-05 04:00:01 [THREAD(6296)] [ERROR] [CDbBase::ExecSql] ---------------------------Exec command VOER.---------------------------
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [CDbBase::ExecSql] Exec sql log: Msg 3201, Level 16, State 1, Server SQLSRV-9, Line 1

    Cannot open backup device

    'C:\Backups\db.db.db.183@aclm_db_imc_aclm_db_20230705_040001_full.db'.

    Operating system error 5(Access is denied.).

    Msg 3013, Level 16, State 1, Server db.db.db.183, Line 1

    BACKUP DATABASE is terminating abnormally.


    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [Remove] Remove file: C:\Program Files\iMC\dbman\bin\dbop.sql.log
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [Remove] Remove file: C:\Program Files\iMC\dbman\bin\dbop.sql
    2023-07-05 04:00:01 [THREAD(6296)] [ERROR] [CDbBackup::BackupOneLocalDBase] Fail to backup 'C:\Backups\db.db.db.183@aclm_db_imc_aclm_db_20230705_040001_full.db'.
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [sendTrapAlert] Send alert message to primary hostIP: xx.xx.xx.123
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [Client::connect_to_server] Starting connect to xx.xx.xx.123: 2810
    2023-07-05 04:00:01 [THREAD(6296)] [INFO] [Client::connect_to_server] Established connection to xx.xx.xx.123: 2810
    2023-07-05 04:00:01 [THREAD(6296)] [WARNING] [CDbBackup::BackupAllDataBase] Backup database: aclm_db failed
    2023-07-05 04:00:01 [THREAD(6296)] [ERROR] [CommonBackup] backup all db failed .
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [Remove] Remove file: C:\Program Files\iMC\dbman\bin\..\..\dbman\etc\LastOperationResult.conf
    2023-07-05 04:00:01 [THREAD(6296)] [DEBUG] [CLorOper::modifyLor] Write file LastOperationResult.conf: Result=Backup failed.
    ErrorReport=Database backup or bulk operation failed.
    Tips=If you cannot obtain the failure reason from the ErrorReport, view the dbman/log/dbman.log and dbman_debug.log files.



  • 2.  RE: IMC Remote DB Backup Failure

    Posted Jul 05, 2023 12:24 PM

    Also, I see these errors in the dmalog. I am not sure what to make of them either.

    2023-07-05 09:42:45 [INFO ] [Thread-1] [com.h3c.imc.deploy.dma.AgentMessageHandler$1::run(811)] Agent start connect to server
    2023-07-05 09:42:45 [INFO ] [ActiveMQ Task-1] [com.h3c.imc.deploy.dma.AgentMessageHandler$1$2::transportResumed(825)] Agent transport resumed
    2023-07-05 09:42:45 [INFO ] [ActiveMQ Task-1] [com.h3c.imc.deploy.dma.AgentMessageHandler::connectStateChanged(980)] Agent-Server connect state changed: Connected
    2023-07-05 09:42:45 [INFO ] [ActiveMQ Task-1] [org.apache.activemq.transport.failover.FailoverTransport::doReconnect(1061)] Successfully connected to tcp://localhost:61626
    2023-07-05 09:42:45 [INFO ] [Thread-1] [com.h3c.imc.deploy.dma.AgentMessageHandler$1::run(847)] Agent connection started
    2023-07-05 09:42:45 [INFO ] [Thread-1] [com.h3c.imc.deploy.dma.AgentMessageHandler::connectStateChanged(980)] Agent-Server connect state changed: Connected
    2023-07-05 09:42:45 [INFO ] [Thread-1] [com.h3c.imc.deploy.dma.AgentMessageHandler$1::run(870)] Create Agent MQ connection completed
    2023-07-05 09:42:45 [INFO ] [main] [com.h3c.imc.deploy.dma.DBManConfiguration::setAgentMessageHandler(319)] this server is Master:false
    2023-07-05 09:42:45 [INFO ] [Thread-10] [com.h3c.imc.deploy.dma.IPConfigFileManager::readAllConfigs(169)] Start loading server-addr.xml configuration file
    2023-07-05 09:42:45 [INFO ] [Thread-10] [com.h3c.imc.deploy.dma.IPConfigFileManager::readAllConfigs(233)] Load server-addr.xml configuration file succeeded
    2023-07-05 09:42:45 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.dma.ComponentDeployInfo::isComponentsConformable(628)] Components conformable check failed: no component deployed
    2023-07-05 09:42:45 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.IPConfigFileManager::readAllConfigs(169)] Start loading server-addr.xml configuration file
    2023-07-05 09:42:45 [INFO ] [main] [com.h3c.imc.deploy.DMALauncher::checkBatchUpgrade(712)] checkBatchUpgrade start
    2023-07-05 09:42:45 [INFO ] [main] [com.h3c.imc.deploy.DMALauncher::checkBatchUpgrade(725)] installInfo.getServerType()=Slave
    2023-07-05 09:42:45 [INFO ] [main] [com.h3c.imc.deploy.DMALauncher::checkBatchUpgrade(726)] -------------
    2023-07-05 09:42:45 [ERROR] [main] [com.h3c.imc.deploy.dma.DeployManager::getInstance(78)] This method only can be used in master
    2023-07-05 09:42:45 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.Util::getCustomImcLogo(3331)] path:C:\Program Files\iMC/deploy/components/common/deploy/custom/images/imc_logo.png
    2023-07-05 09:42:45 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.IPConfigFileManager::readAllConfigs(233)] Load server-addr.xml configuration file succeeded
    2023-07-05 09:42:45 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.dma.DeployerMonitorAgent::load(144)] ??????imc_logo.png
    2023-07-05 09:42:45 [ERROR] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabase(197)] Can not found database.
    2023-07-05 09:42:45 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(111)] checkDatabaseVersion
    2023-07-05 09:42:45 [ERROR] [main] [com.h3c.imc.deploy.DMALauncher::checkBatchUpgrade(869)] null
    java.lang.IllegalStateException: This method only can be used in master
        at com.h3c.imc.deploy.dma.DeployManager.getInstance(DeployManager.java:79) ~[deploy.jar:?]
        at com.h3c.imc.deploy.DMALauncher.checkBatchUpgrade(DMALauncher.java:727) ~[deploy.jar:?]
        at com.h3c.imc.deploy.DMALauncher.loadUI(DMALauncher.java:558) ~[deploy.jar:?]
        at com.h3c.imc.deploy.DMALauncher.main(DMALauncher.java:187) ~[deploy.jar:?]
    2023-07-05 09:42:45 [INFO ] [iMC-Stack-Trace-Printer] [com.h3c.imc.deploy.StackTracePrinter::printStackTrace(71)] thread id: 1, name: main, RUNNABLE 
        java.base@11.0.11/java.lang.SecurityManager.nonExportedPkgs(SecurityManager.java:1191)
        java.base@11.0.11/java.lang.SecurityManager$$Lambda$232/0x0000000800427440.apply(Unknown Source)
        java.base@11.0.11/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        java.base@11.0.11/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        java.base@11.0.11/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        java.base@11.0.11/java.util.Iterator.forEachRemaining(Iterator.java:133)
        java.base@11.0.11/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        java.base@11.0.11/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        java.base@11.0.11/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        java.base@11.0.11/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        java.base@11.0.11/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        java.base@11.0.11/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        java.base@11.0.11/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        java.base@11.0.11/java.lang.SecurityManager.addNonExportedPackages(SecurityManager.java:1158)
        java.base@11.0.11/java.lang.SecurityManager.<clinit>(SecurityManager.java:1143)
        app//org.apache.logging.log4j.util.PrivateSecurityManagerStackTraceUtil.<clinit>(PrivateSecurityManagerStackTraceUtil.java:38)
        app//org.apache.logging.log4j.util.StackLocator.getCurrentStackTrace(StackLocator.java:86)
        app//org.apache.logging.log4j.util.StackLocatorUtil.getCurrentStackTrace(StackLocatorUtil.java:116)
        app//org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:112)
        app//org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:97)
        app//org.apache.logging.log4j.core.impl.MutableLogEvent.getThrownProxy(MutableLogEvent.java:347)
        app//org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:63)
        app//org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:44)
        app//org.apache.logging.log4j.core.layout.PatternLayout$PatternFormatterPatternSerializer.toSerializable(PatternLayout.java:384)
        app//org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:240)
        app//org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:225)
        app//org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:59)
        app//org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:215)
        app//org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:208)
        app//org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:199)
        app//org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:301)
        app//org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
        app//org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
        app//org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
        app//org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
        app//org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:675)
        app//org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:633)
        app//org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:616)
        app//org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:552)
        app//org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
        app//org.apache.logging.log4j.core.Logger.log(Logger.java:161)
        app//org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
        app//org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
        app//org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
        app//org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:1994)
        app//org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1852)
        app//org.apache.logging.log4j.jcl.Log4jLog.error(Log4jLog.java:116)
        app//com.h3c.imc.deploy.DMALauncher.checkBatchUpgrade(DMALauncher.java:869)
        app//com.h3c.imc.deploy.DMALauncher.loadUI(DMALauncher.java:558)
        app//com.h3c.imc.deploy.DMALauncher.main(DMALauncher.java:187)
    thread id: 20, name: ActiveMQ Task-1, TIMED_WAITING, java.util.concurrent.SynchronousQueue$TransferStack@442fb745
        java.base@11.0.11/jdk.internal.misc.Unsafe.park(Native Method)
        java.base@11.0.11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 26, name: ActiveMQ Session Task-1, TIMED_WAITING, java.util.concurrent.SynchronousQueue$TransferStack@4ed001d4
        java.base@11.0.11/jdk.internal.misc.Unsafe.park(Native Method)
        java.base@11.0.11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 33, name: Image Fetcher 1, TIMED_WAITING, java.util.Vector@3e247934
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.desktop@11.0.11/sun.awt.image.ImageFetcher.nextImage(ImageFetcher.java:154)
        java.desktop@11.0.11/sun.awt.image.ImageFetcher.fetchloop(ImageFetcher.java:207)
        java.desktop@11.0.11/sun.awt.image.ImageFetcher.run(ImageFetcher.java:176)
    thread id: 5, name: Attach Listener, RUNNABLE 
    thread id: 24, name: ActiveMQ InactivityMonitor WriteCheckTimer, TIMED_WAITING, java.util.TaskQueue@651de423
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.base@11.0.11/java.util.TimerThread.mainLoop(Timer.java:553)
        java.base@11.0.11/java.util.TimerThread.run(Timer.java:506)
    thread id: 14, name: Java2D Disposer, WAITING, java.lang.ref.ReferenceQueue$Lock@5c31e25a
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.base@11.0.11/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
        java.base@11.0.11/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
        java.desktop@11.0.11/sun.java2d.Disposer.run(Disposer.java:144)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 16, name: AWT-Windows, RUNNABLE 
        java.desktop@11.0.11/sun.awt.windows.WToolkit.eventLoop(Native Method)
        java.desktop@11.0.11/sun.awt.windows.WToolkit.run(WToolkit.java:305)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 29, name: DistMonitor, TIMED_WAITING 
        java.base@11.0.11/java.lang.Thread.sleep(Native Method)
        app//com.h3c.imc.deploy.dma.MonitorPanel$DiskInfoTableModel.run(MonitorPanel.java:1624)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 3, name: Finalizer, WAITING, java.lang.ref.ReferenceQueue$Lock@122186a9
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.base@11.0.11/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
        java.base@11.0.11/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
        java.base@11.0.11/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
    thread id: 30, name: AWT-EventQueue-0, RUNNABLE 
        java.desktop@11.0.11/java.beans.ChangeListenerMap.get(ChangeListenerMap.java:135)
        java.desktop@11.0.11/java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:328)
        java.desktop@11.0.11/java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:266)
        java.desktop@11.0.11/java.awt.Component.firePropertyChange(Component.java:8731)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1176)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Container.updateChildGraphicsData(Container.java:1183)
        java.desktop@11.0.11/java.awt.Component.updateGraphicsData(Component.java:1168)
        java.desktop@11.0.11/java.awt.Component.setGraphicsConfiguration(Component.java:1156)
        java.desktop@11.0.11/java.awt.Container.addImpl(Container.java:1136)
        java.desktop@11.0.11/javax.swing.JLayeredPane.addImpl(JLayeredPane.java:237)
        java.desktop@11.0.11/java.awt.Container.add(Container.java:997)
        java.desktop@11.0.11/javax.swing.JRootPane.setContentPane(JRootPane.java:603)
        java.desktop@11.0.11/javax.swing.JFrame.setContentPane(JFrame.java:679)
        app//com.h3c.imc.deploy.dma.DeployerMonitorAgent.load(DeployerMonitorAgent.java:157)
        app//com.h3c.imc.deploy.DMALauncher$1$1.run(DMALauncher.java:523)
        java.desktop@11.0.11/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
        java.desktop@11.0.11/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)
        java.desktop@11.0.11/java.awt.EventQueue$4.run(EventQueue.java:721)
        java.desktop@11.0.11/java.awt.EventQueue$4.run(EventQueue.java:715)
        java.base@11.0.11/java.security.AccessController.doPrivileged(Native Method)
        java.base@11.0.11/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
        java.desktop@11.0.11/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)
        java.desktop@11.0.11/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
        java.desktop@11.0.11/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
        java.desktop@11.0.11/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
        java.desktop@11.0.11/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
        java.desktop@11.0.11/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
        java.desktop@11.0.11/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
    thread id: 15, name: AWT-Shutdown, WAITING, java.lang.Object@f51b000
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.base@11.0.11/java.lang.Object.wait(Object.java:328)
        java.desktop@11.0.11/sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:291)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 39, name: iMC-Stack-Trace-Printer, RUNNABLE 
        java.base/java.lang.Thread.getStackTrace(Thread.java:1602)
        com.h3c.imc.deploy.StackTracePrinter.printStackTrace(StackTracePrinter.java:66)
        com.h3c.imc.deploy.StackTracePrinter.run(StackTracePrinter.java:43)
    thread id: 18, name: Thread-0, RUNNABLE 
        java.base@11.0.11/java.net.PlainSocketImpl.accept0(Native Method)
        java.base@11.0.11/java.net.PlainSocketImpl.socketAccept(PlainSocketImpl.java:159)
        java.base@11.0.11/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
        java.base@11.0.11/java.net.ServerSocket.implAccept(ServerSocket.java:565)
        java.base@11.0.11/java.net.ServerSocket.accept(ServerSocket.java:533)
        app//com.h3c.imc.deploy.DMALauncher.listenForCommand(DMALauncher.java:1200)
        app//com.h3c.imc.deploy.DMALauncher.access$200(DMALauncher.java:74)
        app//com.h3c.imc.deploy.DMALauncher$3.run(DMALauncher.java:1187)
    thread id: 38, name: iMC-Database-Connect-Check, RUNNABLE 
        java.base@11.0.11/java.lang.ClassLoader.defineClass1(Native Method)
        java.base@11.0.11/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
        java.base@11.0.11/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
        java.base@11.0.11/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
        java.base@11.0.11/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
        java.base@11.0.11/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
        java.base@11.0.11/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
        java.base@11.0.11/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        java.base@11.0.11/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        java.base@11.0.11/java.lang.Class.forName0(Native Method)
        java.base@11.0.11/java.lang.Class.forName(Class.java:315)
        app//com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor.checkDatabaseVersion(DatabaseConnectMonitor.java:140)
        app//com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor.run(DatabaseConnectMonitor.java:95)
    thread id: 25, name: ActiveMQ Task-2, TIMED_WAITING, java.util.concurrent.SynchronousQueue$TransferStack@442fb745
        java.base@11.0.11/jdk.internal.misc.Unsafe.park(Native Method)
        java.base@11.0.11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        java.base@11.0.11/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        java.base@11.0.11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 34, name: Thread-8, TIMED_WAITING 
        java.base@11.0.11/java.lang.Thread.sleep(Native Method)
        app//com.h3c.imc.deploy.dma.DeployPanel$1.run(DeployPanel.java:285)
    thread id: 37, name: Database Checking Thread, TIMED_WAITING 
        java.base@11.0.11/java.lang.Thread.sleep(Native Method)
        app//com.h3c.imc.deploy.dma.EnvPanel$10.run(EnvPanel.java:1171)
    thread id: 28, name: Image Fetcher 0, TIMED_WAITING, java.util.Vector@3e247934
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.desktop@11.0.11/sun.awt.image.ImageFetcher.nextImage(ImageFetcher.java:154)
        java.desktop@11.0.11/sun.awt.image.ImageFetcher.fetchloop(ImageFetcher.java:207)
        java.desktop@11.0.11/sun.awt.image.ImageFetcher.run(ImageFetcher.java:176)
    thread id: 11, name: Common-Cleaner, TIMED_WAITING, java.lang.ref.ReferenceQueue$Lock@569770e1
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.base@11.0.11/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
        java.base@11.0.11/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
        java.base@11.0.11/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
    thread id: 32, name: Thread-7, TIMED_WAITING 
        java.base@11.0.11/java.lang.Thread.sleep(Native Method)
        app//com.h3c.imc.deploy.dma.GraphicMonitorPanel$Surface.run(GraphicMonitorPanel.java:435)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 21, name: ActiveMQ InactivityMonitor ReadCheckTimer, TIMED_WAITING, java.util.TaskQueue@489da054
        java.base@11.0.11/java.lang.Object.wait(Native Method)
        java.base@11.0.11/java.util.TimerThread.mainLoop(Timer.java:553)
        java.base@11.0.11/java.util.TimerThread.run(Timer.java:506)
    thread id: 22, name: ActiveMQ Transport: tcp://localhost/127.0.0.1:61626@58624, RUNNABLE 
        java.base@11.0.11/java.net.SocketInputStream.socketRead0(Native Method)
        java.base@11.0.11/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
        java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:168)
        java.base@11.0.11/java.net.SocketInputStream.read(SocketInputStream.java:140)
        app//org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
        app//org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:634)
        app//org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:59)
        app//org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:619)
        java.base@11.0.11/java.io.DataInputStream.readInt(DataInputStream.java:392)
        app//org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:280)
        app//org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
        app//org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
        app//org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 31, name: Thread-6, TIMED_WAITING 
        java.base@11.0.11/java.lang.Thread.sleep(Native Method)
        app//com.h3c.imc.deploy.dma.GraphicMonitorPanel$Surface.run(GraphicMonitorPanel.java:435)
        java.base@11.0.11/java.lang.Thread.run(Thread.java:829)
    thread id: 4, name: Signal Dispatcher, RUNNABLE 
    thread id: 2, name: Reference Handler, RUNNABLE 
        java.base@11.0.11/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
        java.base@11.0.11/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
        java.base@11.0.11/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)

    2023-07-05 09:42:46 [INFO ] [ActiveMQ Session Task-1] [com.h3c.imc.deploy.dma.AgentMessageHandler::masterConnectStateChanged(1005)] Receive master state change message: Connected
    2023-07-05 09:42:46 [INFO ] [ActiveMQ Session Task-1] [com.h3c.imc.deploy.dma.DeployerMonitorAgent::update(455)] Master connect state changed: Connected
    2023-07-05 09:42:46 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.dma.ComponentDeployInfo::isComponentsConformable(628)] Components conformable check failed: no component deployed
    2023-07-05 09:42:46 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.dma.ComponentDeployInfo::isComponentsConformable(628)] Components conformable check failed: no component deployed
    2023-07-05 09:42:46 [INFO ] [AWT-EventQueue-0] [com.h3c.imc.deploy.dma.ComponentDeployInfo::isComponentsConformable(628)] Components conformable check failed: no component deployed
    2023-07-05 09:42:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(144)] DBTYPE_SQLSERVER.equals(dbType)
    2023-07-05 09:42:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(149)] str: Microsoft SQL Server 2019 (RTM-CU18-GDR) (KB5021124) - 15.0.4280.7 (X64) 
        Jan 23 2023 12:37:13 
        Copyright (C) 2019 Microsoft Corporation
        Standard Edition (64-bit) on Windows Server 2022 Standard 10.0 <X64> (Build 20348: ) (Hypervisor)

    2023-07-05 09:42:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(155)] !str.contains("Evaluation")
    2023-07-05 10:07:52 [WARN ] [AWT-EventQueue-0] [com.h3c.imc.deploy.DeployStateTracer::readState(152)] Can not find deploy state file.
    2023-07-05 10:12:46 [ERROR] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabase(197)] Can not found database.
    2023-07-05 10:12:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(111)] checkDatabaseVersion
    2023-07-05 10:12:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(144)] DBTYPE_SQLSERVER.equals(dbType)
    2023-07-05 10:12:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(149)] str: Microsoft SQL Server 2019 (RTM-CU18-GDR) (KB5021124) - 15.0.4280.7 (X64) 
        Jan 23 2023 12:37:13 
        Copyright (C) 2019 Microsoft Corporation
        Standard Edition (64-bit) on Windows Server 2022 Standard 10.0 <X64> (Build 20348: ) (Hypervisor)

    2023-07-05 10:12:46 [INFO ] [iMC-Database-Connect-Check] [com.h3c.imc.deploy.dma.monitor.DatabaseConnectMonitor::checkDatabaseVersion(155)] !str.contains("Evaluation")
    2023-07-05 10:16:40 [WARN ] [AWT-EventQueue-0] [com.h3c.imc.deploy.DeployStateTracer::readState(152)] Can not find deploy state file.




  • 3.  RE: IMC Remote DB Backup Failure

    Posted Jul 06, 2023 02:49 AM

    Hello,

    I would suggest checking what user is running the MSSQLSERVER service (rather than the agent), and see if they have permissions for the backup directory. You might fix it by setting the MSSQLSERVER service to 'Local System', or at least that worked for me when I had such an issue. The problem there was that the SQL server was running with some special service account that did not have permission to the backup directory.

    Keep in mind that DBMAN is also using the iMC\dbman\bak directory as a temporary storage (on both systems) while it transfers backup files from the remote DB to the local server, so you may need to adjust permissions there as well.



    ------------------------------
    Justin Guse
    ------------------------------

    Original Message


  • 4.  RE: IMC Remote DB Backup Failure

    Posted Jul 06, 2023 11:59 AM

    Thanks Justin for the reply. I should have mentioned that I already checked the SQL service and confirmed the MSSQLSERVER had access to the backup folder location.

    It had been working and just stopped but the notification was not sending. 

    Yesterday, after posting this discussion, I noticed that the remote DB server did not show up in the Monitoring Agent on either the IMC server or the remote DB server so I deployed one of the un-deployed options and it showed up on both and automated backups started working again.  I am not sure what the cause was but its working again, so I can move onto something else.

    Thanks again,

    Matt


    Original Message


Related Content