Issue
StruxureWare Data Center Expert message "Device status may be inaccurate due to attempt to transfer DDF failed"
Product Line
StruxureWare Data Center Expert (DCE)
InfraStruXure Central (ISXC)
APC Networked Devices
Environment
StruxureWare Data Center Expert (all versions)
StruxureWare Central (all versions)
InfraStruXure Central (all versions)
APC Network Management Card (NMC)
Cause
Data Center Expert uses device definition files (DDFs) to understand what information to poll from a discovered device. During the discovery of APC devices and after a firmware update of an APC NMC, Data Center Expert will attempt to download a DDF file (ddf.zip) from the device using either FTP or SCP (SSH). If the server is unable to perform this transfer, you will see a message stating "Device status may be inaccurate due to attempt to transfer DDF failed".
Resolution:
Device Definition File Transfer Process
1) A condition occurs that causes Data Center Expert to reach out to the device to pull down the DDF.
2) Data Center Expert attempts to connect to the device using passive FTP/SCP using credentials and ports defined within: DCE Desktop Client > Device menu > SNMP Device Communication Settings > Device File Transfer Settings.
3) A connection to the device is established using an ephemeral port connection. See Additional Information for a description and example of Ephemeral Ports.
4) The ddf.zip file from the device is transferred from the NMC to DCE.
Checking APC NMC log files to determine possible cause
1) Log into the Data Center Expert Desktop Client using administrator credentials.
2) Within the monitoring perspective, right-click on the device and select Request Device Scan.
NOTE: This step is to ensure that the DCE connection attempt is logged recently in the NMC Event Log.
3) Log into the web interface of the APC NMC using administrator credentials.
4) Logs Menu > Events > Log.
5) There are three different possible options:
Detected an unauthorized user attempting to access the FTP/SCP interface from DCE_IP_Address.
DCE attempted to log into the device using credentials that are not defined within the NMC as administrator. Follow the steps below for verifying DCE Device File Transfer Settings.
No entries for FTP/SCP access
DCE was unable to make a connection to the NMC over FTP/SCP. This is due to one of the following:
- FTP/SCP may not be enabled on the device or ports may be incorrect, see section Enabling FTP/SCP on APC NMC.
- Specified FTP/SCP ports are blocked on the network, see section Network Firewall or ACL.
FTP/SCP user 'username' logged in from IP_Address / FTP/SCP user 'username' logged out from DCE_IP_Address
DCE was able to make a successful connection to the device over the specified FTP/SCP port, but was not able to establish a passive FTP/SCP connection to the device. For additional information on passive connections over ephemeral ports, see section Network Firewall or ACL.
Enabling FTP/SCP on APC NMC
NOTE: Only FTP or SCP need to be enabled for Data Center Expert to retrieve the essentials alerts device definition file.
After FTP/SCP settings are verified, correct the credentials listed within Data Center Expert. For instructions see section DCE Device File Transfer Settings.
FTP NMC firmware version 6.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Configuration Menu > Network > FTP Server.
3) Ensure that the Enable checkbox is checked. Also take note of the port number, by default it is 21.
SCP NMC firmware version 6.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Configuration Menu > Network > Console > Access.
3) Ensure that the Enable checkbox is checked for SSH. Also take note of the port number, by default it is 22.
FTP NMC firmware version 3.X-5.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Administration Tab > Network Sub-tab > FTP Server.
3) Ensure that the Enable checkbox is checked. Also take note of the port number, by default it is 21.
SCP NMC firmware version 3.X-5.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Administration Tab > Network Sub-tab > Console > Access.
3) Enable an option for SSH. Also take note of the port number, by default it is 22.
DCE Device File Transfer Settings
NOTE: It is recommended that as few entries as possible apply to any one single device within the device file transfer settings.
1) Log into the Data Center Expert desktop client using administrator credentials.
2) Device menu > SNMP Device Communication Settings > Device File Transfer Settings.
3) Ensure that there is an entry created in this screen that applies to the device with the settings from the APC NMC. Note that credentials provided must be the NMC Administrator account credentials.
NOTE: Wildcards and dashes can be used to specify ranges, for example: *.*.*.* (all devices), 10.10.*.*, 10.10.17.1-20, 10.10.10-17.*
NOTE: If all of your devices use the same username/password, you will only need one entry within the Device File Transfer Settings screen. If every device uses a unique set of administrator credentials, an entry will need to exist for each device within the Device File transfer settings screen.
Network Firewall or ACL (Access Control Lists)
If FTP/SCP is fully blocked on the network between DCE and the NMC there will be no events within the NMC Event Logs. This is because the connection request never gets to the NMC across the network. You must work with your local network administrator to unblock the required ports and allow passive connections over ephemeral ports, see Additional Information for a description and example of Ephemeral Ports.
If there are successful login/logout messages from DCE within the NMC Event Logs, the connection over the specified FTP/SCP ports is not blocked on the network.
Additional Information & Troubleshooting:
Data Canter Expert 7.8.1 or earlier trying to discover an NMC3
- The NMC3 or a device with an embedded NMC3 when configured for SSH default to an ECDSA algorithm which DCE 781 and earlier does not support.
- In this case the two recommended options are to either
1) Upgrade to 7.9.0 or later since these versions support ECDSA (preferred)
2) Delete the ECDSA key on the NMC3 (firmware 1.4.0.19 or newer) and generate an RSA one instead by using the following 2 commands from a CLI connection to the card,
ssh key -d
ssh key -rsa 2048
My company is unable to allow FTP/SCP access between DCE and the devices
- FTP is insecure and blocked on most internal networks. If FTP is not avaialble for use on your network, try utilizing SCP.
- There is currently no way to disable the alarm or manually import the essential alerts DDF file from the device into DCE.
Rack ATS AP44XX DDF transfer issue still occurs after troubleshooting steps listed above
- It has been seen in the AP44XX series Rack ATS firmware (ats4g 6.4.7 or 6.5.0) that this message may occur due to a firmware related issue. If you have this version of firmware and see this error, please upgrade to the most recent firmware (AOS 6.5.0 with an ats4g 6.5.1). This firmware is available for download on the APC web site as well as in the newest StruxureWare DCE firmware catalog.
Ephemeral Ports are blocked on the Network
- A PASSIVE FTP connection is established over an ephemeral port to allow multiple FTP/SCP sessions to occur at the same time.
- Ephemeral ports are ports that range from 1024 to 65535.
- Below is a table containing the contents of a network packet capture to show the use of the ephemeral ports:
- Packets 1-5 Establishing a connection from DCE Ephemeral port to NMC FTP port 21 (22 if SSH is being used).
- Packets 6-9: Logging in using Username and Password defined in DCE Device File Transfer Settings.
- Packets 10-19: Requesting size of ddf.zip and passive ftp connection.
- Packets 20-39: Transferring file from NMC to DCE.
- Packets 40-44: Closing Connection.
StruxureWare Data Center Expert message "Device status may be inaccurate due to attempt to transfer DDF failed"
Product Line
StruxureWare Data Center Expert (DCE)
- Basic Appliance (AP9465)
- Standard Appliance (AP9470)
- Enterprise Appliance (AP9475)
- Virtual Appliance (AP94VMACT)
StruxureWare Central (SWC)- Standard Appliance (AP9470)
- Enterprise Appliance (AP9475)
- Virtual Appliance (AP94VMACT)
InfraStruXure Central (ISXC)
APC Networked Devices
Environment
StruxureWare Data Center Expert (all versions)
StruxureWare Central (all versions)
InfraStruXure Central (all versions)
APC Network Management Card (NMC)
Cause
Data Center Expert uses device definition files (DDFs) to understand what information to poll from a discovered device. During the discovery of APC devices and after a firmware update of an APC NMC, Data Center Expert will attempt to download a DDF file (ddf.zip) from the device using either FTP or SCP (SSH). If the server is unable to perform this transfer, you will see a message stating "Device status may be inaccurate due to attempt to transfer DDF failed".
Resolution:
Device Definition File Transfer Process
1) A condition occurs that causes Data Center Expert to reach out to the device to pull down the DDF.
2) Data Center Expert attempts to connect to the device using passive FTP/SCP using credentials and ports defined within: DCE Desktop Client > Device menu > SNMP Device Communication Settings > Device File Transfer Settings.
3) A connection to the device is established using an ephemeral port connection. See Additional Information for a description and example of Ephemeral Ports.
4) The ddf.zip file from the device is transferred from the NMC to DCE.
Checking APC NMC log files to determine possible cause
1) Log into the Data Center Expert Desktop Client using administrator credentials.
2) Within the monitoring perspective, right-click on the device and select Request Device Scan.
NOTE: This step is to ensure that the DCE connection attempt is logged recently in the NMC Event Log.
3) Log into the web interface of the APC NMC using administrator credentials.
4) Logs Menu > Events > Log.
5) There are three different possible options:
Detected an unauthorized user attempting to access the FTP/SCP interface from DCE_IP_Address.
DCE attempted to log into the device using credentials that are not defined within the NMC as administrator. Follow the steps below for verifying DCE Device File Transfer Settings.
No entries for FTP/SCP access
DCE was unable to make a connection to the NMC over FTP/SCP. This is due to one of the following:
- FTP/SCP may not be enabled on the device or ports may be incorrect, see section Enabling FTP/SCP on APC NMC.
- Specified FTP/SCP ports are blocked on the network, see section Network Firewall or ACL.
FTP/SCP user 'username' logged in from IP_Address / FTP/SCP user 'username' logged out from DCE_IP_Address
DCE was able to make a successful connection to the device over the specified FTP/SCP port, but was not able to establish a passive FTP/SCP connection to the device. For additional information on passive connections over ephemeral ports, see section Network Firewall or ACL.
Enabling FTP/SCP on APC NMC
NOTE: Only FTP or SCP need to be enabled for Data Center Expert to retrieve the essentials alerts device definition file.
After FTP/SCP settings are verified, correct the credentials listed within Data Center Expert. For instructions see section DCE Device File Transfer Settings.
FTP NMC firmware version 6.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Configuration Menu > Network > FTP Server.
3) Ensure that the Enable checkbox is checked. Also take note of the port number, by default it is 21.
SCP NMC firmware version 6.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Configuration Menu > Network > Console > Access.
3) Ensure that the Enable checkbox is checked for SSH. Also take note of the port number, by default it is 22.
FTP NMC firmware version 3.X-5.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Administration Tab > Network Sub-tab > FTP Server.
3) Ensure that the Enable checkbox is checked. Also take note of the port number, by default it is 21.
SCP NMC firmware version 3.X-5.X
1) Log into the web interface of the APC NMC using administrator credentials.
2) Administration Tab > Network Sub-tab > Console > Access.
3) Enable an option for SSH. Also take note of the port number, by default it is 22.
DCE Device File Transfer Settings
NOTE: It is recommended that as few entries as possible apply to any one single device within the device file transfer settings.
1) Log into the Data Center Expert desktop client using administrator credentials.
2) Device menu > SNMP Device Communication Settings > Device File Transfer Settings.
3) Ensure that there is an entry created in this screen that applies to the device with the settings from the APC NMC. Note that credentials provided must be the NMC Administrator account credentials.
NOTE: Wildcards and dashes can be used to specify ranges, for example: *.*.*.* (all devices), 10.10.*.*, 10.10.17.1-20, 10.10.10-17.*
NOTE: If all of your devices use the same username/password, you will only need one entry within the Device File Transfer Settings screen. If every device uses a unique set of administrator credentials, an entry will need to exist for each device within the Device File transfer settings screen.
Network Firewall or ACL (Access Control Lists)
If FTP/SCP is fully blocked on the network between DCE and the NMC there will be no events within the NMC Event Logs. This is because the connection request never gets to the NMC across the network. You must work with your local network administrator to unblock the required ports and allow passive connections over ephemeral ports, see Additional Information for a description and example of Ephemeral Ports.
If there are successful login/logout messages from DCE within the NMC Event Logs, the connection over the specified FTP/SCP ports is not blocked on the network.
Additional Information & Troubleshooting:
Data Canter Expert 7.8.1 or earlier trying to discover an NMC3
- The NMC3 or a device with an embedded NMC3 when configured for SSH default to an ECDSA algorithm which DCE 781 and earlier does not support.
- In this case the two recommended options are to either
1) Upgrade to 7.9.0 or later since these versions support ECDSA (preferred)
2) Delete the ECDSA key on the NMC3 (firmware 1.4.0.19 or newer) and generate an RSA one instead by using the following 2 commands from a CLI connection to the card,
ssh key -d
ssh key -rsa 2048
My company is unable to allow FTP/SCP access between DCE and the devices
- FTP is insecure and blocked on most internal networks. If FTP is not avaialble for use on your network, try utilizing SCP.
- There is currently no way to disable the alarm or manually import the essential alerts DDF file from the device into DCE.
Rack ATS AP44XX DDF transfer issue still occurs after troubleshooting steps listed above
- It has been seen in the AP44XX series Rack ATS firmware (ats4g 6.4.7 or 6.5.0) that this message may occur due to a firmware related issue. If you have this version of firmware and see this error, please upgrade to the most recent firmware (AOS 6.5.0 with an ats4g 6.5.1). This firmware is available for download on the APC web site as well as in the newest StruxureWare DCE firmware catalog.
Ephemeral Ports are blocked on the Network
- A PASSIVE FTP connection is established over an ephemeral port to allow multiple FTP/SCP sessions to occur at the same time.
- Ephemeral ports are ports that range from 1024 to 65535.
- Below is a table containing the contents of a network packet capture to show the use of the ephemeral ports:
- Packets 1-5 Establishing a connection from DCE Ephemeral port to NMC FTP port 21 (22 if SSH is being used).
- Packets 6-9: Logging in using Username and Password defined in DCE Device File Transfer Settings.
- Packets 10-19: Requesting size of ddf.zip and passive ftp connection.
- Packets 20-39: Transferring file from NMC to DCE.
- Packets 40-44: Closing Connection.
Released for:Schneider Electric Indonesia
