Parsing the ridiculously large DNC file – .NET Edition

Posted on by

So previously I posted about a problem one of our customers had parsing the DNC. The solution was done in perl and you can read about it in the Perl Edition of this. And it worked rather well and very quickly. However it presented a support issue: They didn’t have a linux box nor the inclination to install perl on a windows box just for that meaning each month we needed to parse it for them.

So I rewrote it in Window using .NET…

The process is very similar to the perl implementation. The biggest differences lie in the Windows specific coding that need to surround the main processing. See, unlike a command line application on linux, a windows application, at least one using windows forms, needs to afford the user some respect and not be unresponsive while processing is going on. That means threads.

Traditionally, threads in Windows has been essential as well as a pain. But I found a new-ish class that Microsoft put in called the BackgroundWorker that really did the trick here. It allowed me to run the processing in the background while getting status updates as needed to keep the user informed. Really worked out well.

I started out in the graphical side, and created a simple form with just the essentials. The key to good user interaction design is to keep the users goal in mind and remove all excise between where they are and where they want to be.

So, we have a place to specify the DNC file including a browse button, a Start button, a progress bar, and (you can’t see them) some blank labels for status updates. Also on this screen, I pulled over a BackgroundWorker and called it bwSlicer and a OpenFileDialog object and called it ofdDNC.

Not much to it, but that’s the idea.

I link the browse button to the opening of the OpenFileDialog box. I link the value of said dialog to the text box by the browse button so that when one changes, the other changes. I did not implement any kind of save so it remembers the value between uses, which would be the next thing I would do if I worked on this again.

private void btnBrowse_Click(object sender, EventArgs e)
        {
            ofdDNC.FileName = tbDNCFile.Text;
            if (ofdDNC.ShowDialog() == DialogResult.OK)
            {
                tbDNCFile.Text = ofdDNC.FileName;
            }
        }

The Start Button I link up so that it starts the worker thread if it isn’t already running, and cancels it if it is. I also change the name of the button to reflect the next action it would take. I am also making the progress bar appear and disappear based on what the worker thread is doing. So that happens here too.

private void button2_Click(object sender, EventArgs e)
        {
            if (button2.Text == "Start")
            {
                lblResults.Text = "Starting...";
                button2.Text = "Cancel";
                progressBar1.Visible = true;
                bwSlicer.RunWorkerAsync(ofdDNC.FileName);
            }
            else
            {
                bwSlicer.CancelAsync();
            }
        }

The BackgroundWorker object has several events I subscribed to. The simplest is the ProgressChanged event. With this one we just change the progress bar. The progress I am going to do in a kind of hokey way, since I don't want to read the whole DNC file and then do the math to determine at which point I am at or anything like that so I am just going by the first two numbers of the area code of the current line of processing. So when I am processing area code 834, it's going to be at 83%. This isn't perfect and would be the second thing I would look at if reworking this, but it's simple and does the job. Again, since the DNC is in numeric order, this works.

private void bwSlicer_ProgressChanged(object sender, ProgressChangedEventArgs e)
        {
            progressBar1.Value = e.ProgressPercentage;
            lblResults.Text = e.ProgressPercentage.ToString() + "% (" + e.UserState.ToString() + ")";
        }

The RunWorkerCompleted event is going to be the next easiest. This one we check the status that the worker end with. If there was an error, print the error in that label we have. If cancelled, say so, and if completed successfully, say so. Also set the progress bar to 100% and change the button back to Start.

private void bwSlicer_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
            if (e.Error != null)
            {
                lblResults.Text = "Error: " + e.Error.Message;
            }
            else if (e.Cancelled)
            {
                lblResults.Text = "User Cancelled";
            }
            else
            {
                lblResults.Text = "100%";
            }

            progressBar1.Value = 100;
            button2.Text = "Start";
        }

The DoWork event delegate is where the work is all done. Again, this is fairly similar to the process as it was done in Perl except we have to specify things a bit more with the strongly typed .NET language. We first check to make sure the file exists, then change to the directory where the file is located. We open the file by creating a StreamReader with it. We go through the file and write to the appropriate file for the line as well as report progress as it comes along. Once done we switch back to the original directory and end.

private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e) 
        {
            BackgroundWorker w = sender as BackgroundWorker;

            string filename = e.Argument.ToString();
            if (File.Exists(filename))
            {
                string oldDir = Environment.CurrentDirectory;
                Environment.CurrentDirectory = filename.Substring(0, filename.LastIndexOf('\'));
                
                using (StreamReader srDNCFile = new StreamReader(filename))
                {
                    
                    string line = "";
                    string num = "0";
                    string start = "";
                    int percent = 0;
                    int tmpPer = 0;
                    string tmpZip = "";
                    string zip = "";

                    StreamWriter swFile = new StreamWriter(num + ".txt"); 

                    do
                    {
                        line = srDNCFile.ReadLine();
                        start = line.Substring(0, 1);
                        tmpZip = line.Substring(0, 6);

                        try {
                            tmpPer = int.Parse(line.Substring(0, 2));
                        } catch(Exception ex) {
                            continue;
                        }
                        
                        if (start != num)
                        {
                            swFile.Close();
                            if (File.Exists(start + ".txt"))
                            {
                                File.Delete(start + ".txt");
                            }
                            swFile = new StreamWriter(start + ".txt");
                            num = start;
                        }
                        swFile.WriteLine(line);
                        if (zip != tmpZip)
                        {
                            zip = tmpZip;
                            percent = tmpPer;
                            w.ReportProgress(percent, zip);
                        }

                    } while (srDNCFile.Peek() != -1);

                    swFile.Close();
                }

                Environment.CurrentDirectory = oldDir;
            }
            else
            {
                throw new FileLoadException("File does not exist", filename);
            }

            e.Result = "Done";
        }

And that's it! The .NET Edition of the DNC parser.

Leave a Reply