Task hangs while executing process (BCP) RRS feed

  • Question

  • Hey guys,

    We have a HPC node that runs some of our tasks in it. I have a task in my .net project that kicks the bcp utility on the HPC node and the output of the query that I have runs into 9 Mb.

    When the HPC node runs this task the output of the query is dumped into a file and then after it dumps around 5mb of data it suddenly stops dumping any more data and this happens all the time. (Please note this isnt any data issue as its not crashing on a particular row every time). this may or may not be of significance but I dump the data into a different server which has adequate permissions set.

    I have run the command with the same query directly on the hpc node and on other comps and it gives the right output.

    I'm running the bcp command as follows:

    var processInfo = new ProcessStartInfo("bcp.exe", argument) { RedirectStandardOutput = true, RedirectStandardError = true, CreateNoWindow = true, UseShellExecute = false };

            var proc = new Process { StartInfo = processInfo, EnableRaisingEvents = true };
    .Exited += new EventHandler(bcp_log);

    So my code actually waits for each bcp task to run before it goes ahead as I call it multiple times.

    FYI to remind you again it only fails when my o/p exceeds a certain no of bytes in this case approx 5mb.

    Any help is much appreciated.


    P.S: I would want to add the bcp utility is installed on all the hpc nodes

    Tuesday, May 25, 2010 10:14 PM


  • Hi Anil,

    It's not clear whether this is a problem with HPC or with the way you're using the bcp utility. You mentioned that you can successfully run the bcp command manually. However, you might like to try running your actual HPC task commandline manually (i.e. the task commandline which runs your "new Process" code to start bcp running). It may be a problem related to the way you are referencing the bcp executable when creating the instance of ProcessStartInfo.

    I also came across a similar situation on the following forum thread: http://social.msdn.microsoft.com/forums/en-us/sqlintegrationservices/thread/3FEB599A-D1A3-4153-812C-A4A01224F9AB



    Wednesday, June 2, 2010 8:56 PM