1

I have this program that has 3 processes:

  • P1 - reads data from a file does some processing and writes data to a new file .
  • P2 reads from this file does some processing and writes data to another file.
  • P3 reads data from this file and does some processing and prints an output.

Right now these are sequential but I dont need P1 to write all the data before P2 can start reading. So id like to make all 3 processes run in parallel in java. So that as and when P1 writes a line to the file, P2 can start reading it and the same for the file that P2 writes to and P3 reads from.

Any ideas how I can implement this in java?

Here is an image description of what I'm trying to do: link to image

philipvr
  • 5,738
  • 4
  • 32
  • 44
Vandit Aruldas
  • 11
  • 1
  • 1
    This is far easier to implement in one process. There is no standard way to read a file while it is being written to, you can invent one (which is not as easy as it sounds), or you can just use multiple threads in one process and pass the data via a queue. – Peter Lawrey Apr 04 '17 at 14:58
  • if you are not using the file created by p1/p2 again, why do you need to store it? Just create a pipeline with a queue of strings/lines. If you go through the file system it would become more convoluted – bichito Apr 04 '17 at 15:04
  • Sounds like a case where RxJava might be helpful. Making the writer observable and let the reader subscribe to changes. – cYrixmorten Apr 04 '17 at 15:12
  • I read that Files parallel stream works well. You can have P1 run in parallel, then P2, then p3. – Nick Ziebert Apr 04 '17 at 15:12
  • If you want to do this correctly, you should not use `Files`, but `Pipes`. Open files have a highly chaotic behavior because of how the OS and cache handle them. – Guillaume F. Apr 04 '17 at 15:37
  • Looks like this as been answered before. Check this out - http://stackoverflow.com/a/32783641/2254148 – Amer Apr 04 '17 at 15:47
  • I dont have to use files. I dont have much idea about pipes. Could you please guide me to a resource to help me understand how to use pipes in java. Can i use streams or buffers? – Vandit Aruldas Apr 05 '17 at 16:37

0 Answers0