Unidentified memory leak with -prof/-hc
GHC-7.4.2 / binary-0.5.1.0
GHC-7.6.2 / binary-0.5.1.1
Alan confirms that with the development branch of d-p, this leak goes away.
Now the trouble is whether it goes away in the remote case or not.
Not with the development branch of distributed-process though!
The attached profile report (server.ps) was produced by running https://github.com/haskell-distributed/distributed-process-platform/blob/misc-fixes/regressions/LeakByteStrings.hs, which uses `handleCall' and does not appear to be leaking memory - afaict anyway. The second was run with this instead:
runProcess node $ doWork True False – use async, don't kill the server afterwards
runProcess node $ doWork False False – don't use async, don't kill...
and that produced async.ps, attached. That also doesn't look like what we were seeing before.
My working hypothesis is that the change in heap behaviour is due to commit d9fcd8d in distributed-process (optimising local sends to avoid hitting the network), but we'll see. Quite frankly, I'm rather baffled by the changing results. I guess this could mean that there was some leak when using the network-transport/socket layer to communicate with processes on the same node. Or it could manifest again with processes on different nodes - now I'm going to have to take your advice about profiling calls between different executables. I'm also completely bemused why
(a) killing the server process made any difference before (but doesn't now) and
(b) using "bare send" versus Async/handleCall made any difference if the leak was somehow situated below d-p-platform in d-process
I think from here, we need to take some steps:
1. confirm whether or not these profiles (attached) look any better
2. profile two local nodes running in the same executable (is this a leak in the node controller, event loop listener or some such?)
3. profile two nodes running in discrete executables (one client, one server)
And AsyncSTM probably does too - see that attached profiles. Code to reproduce is in LeakByteString.hs (see ./regressions) and the topic thread on parallel-haskell for more details...