Device agent seems to be hung on mssql-plus node

daveo · June 9, 2023, 7:16pm

I have been struggling with the device-agent as of this morning. It was working fine yesterday. I created a new device agent instance and it worked for about five minutes then started hanging when connecting to the sql database.

Output from Device log:

6/9/2023 12:03:06 PM

[info]

Version: 0.8.0

6/9/2023 12:03:06 PM

[info]

Mode: Device Mode

6/9/2023 12:03:06 PM

[info]

Device: R85g4kW3Oy

6/9/2023 12:03:06 PM

[info]

ForgeURL: https://app.flowforge.com

6/9/2023 12:03:06 PM

[info]

Instance : 16458603-6248-4ec8-aa42-c8231f1dc51e

6/9/2023 12:03:06 PM

[info]

Snapshot : 1wd3RGbzr2

6/9/2023 12:03:06 PM

[info]

Settings : fb72ee7bbb2759c6356c86fc68738a84d46a39ab9d63ecb33a13dacb5d1f0636

6/9/2023 12:03:06 PM

[info]

Operation Mode: autonomous

6/9/2023 12:03:06 PM

[info]

Starting MQTT heartbeat thread. Interval: 30s (±5s)

6/9/2023 12:03:08 PM

[info]

Starting Node-RED

6/9/2023 12:03:09 PM

[info]

Welcome to Node-RED ===================

6/9/2023 12:03:09 PM

[info]

Node-RED version: v3.0.2

6/9/2023 12:03:09 PM

[info]

Node.js version: v18.12.1

6/9/2023 12:03:09 PM

[info]

Windows_NT 10.0.19044 x64 LE

6/9/2023 12:03:10 PM

[info]

Loading palette nodes

6/9/2023 12:03:12 PM

[warn]

6/9/2023 12:03:12 PM

[warn]

[@flowforge/nr-file-nodes/file] ‘file in’ already registered by module node-red

6/9/2023 12:03:12 PM

[warn]

6/9/2023 12:03:12 PM

[info]

Settings file : C:\opt\flowforge-device\project\settings.js

6/9/2023 12:03:12 PM

[info]

Context store : ‘memory’ [module=memory]

6/9/2023 12:03:12 PM

[info]

Context store : ‘persistent’ [module=localfilesystem]

6/9/2023 12:03:12 PM

[info]

User directory : C:\opt\flowforge-device\project

6/9/2023 12:03:12 PM

[warn]

Projects disabled : set editorTheme.projects.enabled=true to enable

6/9/2023 12:03:12 PM

[info]

Flows file : C:\opt\flowforge-device\project\flows.json

6/9/2023 12:03:12 PM

[info]

Server now running at http://127.0.0.1:1886/device-editor/

6/9/2023 12:03:12 PM

[info]

Starting flows

6/9/2023 12:03:12 PM

[info]

Started flows

6/9/2023 12:03:13 PM

[info]

Project Link nodes connected

6/9/2023 12:03:22 PM

(node:312632) [DEP0123] DeprecationWarning: Setting the TLS ServerName to an IP address is not permitted by RFC 6066. This will be ignored in a future version.

6/9/2023 12:03:22 PM

(Use node --trace-deprecation ... to show where the warning was created)

6/9/2023 12:03:22 PM

[info]

Connected to server : 10.0.240.37, database : WENCO_Wolverine, port : 1433, user : sastrong text

daveo · June 9, 2023, 7:26pm

I tried the same node with the same sql server on a local hosted version of NR and it works fine.

daveo · June 9, 2023, 7:57pm

I tried a different SQL node in a new snapshot and again it worked for a couple of minutes then the device agent started timing out. The device log isn’t showing any errors.

6/9/2023 12:43:13 PM

[info]

Starting Node-RED

6/9/2023 12:43:14 PM

[info]

Welcome to Node-RED ===================

6/9/2023 12:43:14 PM

[info]

Node-RED version: v3.0.2

6/9/2023 12:43:14 PM

[info]

Node.js version: v18.12.1

6/9/2023 12:43:14 PM

[info]

Windows_NT 10.0.19044 x64 LE

6/9/2023 12:43:16 PM

[info]

Loading palette nodes

6/9/2023 12:43:18 PM

[warn]

------------------------------------------------------

6/9/2023 12:43:18 PM

[warn]

[@flowforge/nr-file-nodes/file] 'file in' already registered by module node-red

6/9/2023 12:43:18 PM

[warn]

------------------------------------------------------

6/9/2023 12:43:18 PM

[info]

Settings file : C:\opt\flowforge-device\project\settings.js

6/9/2023 12:43:18 PM

[info]

Context store : 'memory' [module=memory]

6/9/2023 12:43:18 PM

[info]

Context store : 'persistent' [module=localfilesystem]

6/9/2023 12:43:18 PM

[info]

User directory : C:\opt\flowforge-device\project

6/9/2023 12:43:18 PM

[warn]

Projects disabled : set editorTheme.projects.enabled=true to enable

6/9/2023 12:43:18 PM

[info]

Flows file : C:\opt\flowforge-device\project\flows.json

6/9/2023 12:43:18 PM

[info]

Server now running at http://127.0.0.1:1886/device-editor/

6/9/2023 12:43:18 PM

[info]

Starting flows

6/9/2023 12:43:18 PM

[info]

Started flows

6/9/2023 12:43:20 PM

[info]

Project Link nodes connected

daveo · June 9, 2023, 8:28pm

When I use the device editor, I can see that it is working as expected, the database is being queried and returning data to the node but in the FF flow the device-agent node is timing out.

daveo · June 9, 2023, 9:05pm

I think I have it working again. I think there were two nodes creating a race condition for the SQL device agent. I have removed the device-agent node I think was causing the problem and restarted the instance using the device nodes. Has been running for about 30 minutes now without timing out.

daveo · June 12, 2023, 10:58pm

I am getting this message when I initiate a call to the device-agent: “Error: client not initialised”

What does this mean? I can open the device agent editor but it has lost communications with the flow that calls the device.

Steve · June 13, 2023, 6:17am

Hi @daveo I’m an not following exactly what you mean by “when I initiate a call to the device-agent”

Can you provide a bit more detail please?

daveo · June 13, 2023, 12:46pm

I have a device agent running on a laptop. The laptop has access to a SQL database. The flow in Flowforge NR editor

sends a query to the device agent using the “project call” node. When I look at the device agent flow in the editor, I can see the query being received. The query is sent to a MSSQL node and a debug node is showing the SQL data being returned in the device editor. That data is not being returned to the flow in Flowforge, it is timing out. When I restart the Flowforge flow, it works for a few minutes then starts timing out again. The attached images show how the project links are configured.

I know I have configured the “project-in” node to use the first option, “Receive messages sent to this instance”, but it keeps reverting back to the configuration shown.

Maybe I didn’t save/update the snapshot for this device correctly before? If it is updated in the editor does it also have to be updated in the Flowforge instance?

I will reconfigure the project-in node again and explicitly update the snapshot in FF.

rob · June 13, 2023, 4:31pm

Hello Dave

I don’t think the problem is caused by the snapshots, could I ask you add me to your team and I will take a look?

Thanks

Steve · June 13, 2023, 4:43pm

Hi, I agree with @rob, this is a “care point” / “awareness thing” with using link-call for devices.

Link call works great for remote subroutine situations but only when there is one return.

Since you are deploying to a device, there are two returns (the project instance and your device - and perhaps more depending on how many devices are assigned and running).

It can be done but you must take care to only “return” from one device.

It is not so difficult to achieve using environment variables (to ensure the targeted device is the one that returns). But in this case it may make more sense to simply have two pairs of “project in” and “project out” nodes instead of link call nodes.

Rob can certainly assist you if you respond to his request.

daveo · June 13, 2023, 5:23pm

Thanks @Steve @rob is currently looking into this and has access.
There is only one device per dashboard instance. In this case, there is only one database being accessed by the device as well.

rob · June 13, 2023, 8:50pm

I’m pretty sure the issue is resolved. To make this thread (hopefully) useful for other users in the future…

The issue was that there was a copy of the devices flows running in FlowForge Cloud as well as on the device. The device has network access to specific equipment. When the main flow was requesting data from the device it was in fact getting a reply back from the flows running on FlowForge Cloud, the reply therefore was showing a timeout error.

There are two ways to avoid this outcome.

Suspend the copy of the device flows on FlowForge Cloud so that the only replies come from the device.
Add logic into your flows to identify if the reply is coming from a device or from an instance of Node-RED running in FlowForge Cloud. This can be done by checking the FF_DEVICE_ID in a switch node, if the value returned is blank the system which replied was running in FlowForge Cloud and that reply should be ignored.

Credit to @Steve for spotting what the issue was.

daveo · June 13, 2023, 10:13pm

Thank you @rob and @Steve for quickly resolving this, very much appreciated!

Topic		Replies	Views
Device Agent Update Support device-agent	4	342	June 17, 2023
Node-red-contrib-s7 not working with Node-RED run from the device agent or cloud instance General	9	356	October 31, 2023
FlowForge with Mosquitto MQTT Broker General	9	600	December 22, 2023
'ERR_REQUIRE_ESM error on starting device agent General	2	274	September 29, 2023
Deploy failure in FlowFuse General	4	187	March 28, 2024

Device agent seems to be hung on mssql-plus node

Related topics