1

So I have this idea i'm playing with to make my own neural network experiment from the ground up as a fun hobby exercise, so I have something to learn, and to expand upon in doing research for what I want to code. I'm totally aware i'm re-inventing the wheel. But I'm doing it for learning/fun/hobby purposes.

My approach will not be one with the traditional chance calculation and updating status in the next generation where it will improve on itself, i'm trying something else, that will allow my network to adapt to changing situations, just to see what will happen when I turn off the lights.

What would be a good way to handle the multitude of worker processes my app will end up with?

  1. Should I iterate them on each "tick" and update their state?
  2. Should I put them in threads so they call all function simultaneously(or how important the thread scheduler finds them)?
  3. Should I cluster them in groups and put the groups in threads that iterate them independently.
  4. Other...

My problem with scenario 1 is: That if there are a lot of neurons that it could take a while to update them all, making the overall progress slower. Things that pop to mind is adding update flags to them and only to iterate those for updating. I also could use java streams so java can do it's magic under the hood by making the iteration parallel when possible.

My problem with scenario 2 is: There are only so many threads. On normal cpu's 1 to 8 threads are usually running and it has to host an OS too. So not much wiggle room there. I could expand to using GPU cores, but there's also a variable amount there, and they are only for specialized instructions. Also the same problem as in the traffic jam issue in scenario 3, but on smaller scales, and I want to prevent trashing the thread scheduler

My problem with scenario 3 is: That it causes a "buildup/traffic jam" of signals that need to be processed that cross from cluster A to cluster B if cluster B doesn't get as much time from the thread scheduler as cluster A.

My problem with scenario 4 is: I haven't come up with it yet.

My target language will be Java. I am sure there are solutions for this and similar problems, where lots of little processes have to do a thing to complete one big thing(what comes to mind is 3d game rendering, hence my scenario 1).

I'm leaning towards scenario 1, but what are the things I should consider with all these little workers and what would be the most advisable approach.

Improve this question
1
  • If threads are mostly busy with purely computational tasks, it would not make much sense to introduce higher degree of parallelism than physically supported by hardware. Commented Mar 23, 2018 at 18:04

1 Answer 1

Reset to default
2

You should definitely avoid creating a thread per worker. Having many threads will create a lot of overhead and will drag down your performance. If you want to have a lot of workers, your application will grind down to a near standstill.

The optimal number of threads for your application is something you will need to experiment with but it's generally going to be roughly like the number of CPUs on your system. So if you have 4 hyperthread cores, it might be 7 threads.

To manage this in Java you can either use the Executors framework or use the newer streaming API. Using something like a fixed thread pool executor service will give you a lot of control over this and using the streaming APIs will be more automagical. I tend towards Executors since I have already invested the time in understanding them but if you want to get something going really quickly, you might want to go with the streams.

Improve this answer
6
  • Okay. I though so along those lines too, but wasnt sure if I'd miss something. Would you recommend the clustered sections approach then or the big list approach? Commented Mar 23, 2018 at 22:18
  • I guess I'm not sure if you've got a good way to prioritize the work. What's the clustering strategy? Also, how many CPUs are you running? Commented Mar 26, 2018 at 13:30
  • I have 2 pc's at my disposal. One is a dual core, the other is a quad core. I was thinking in clustering per function. At first just input processing and output actions. later possibly more types of input and output. Commented Mar 26, 2018 at 13:32
  • Is this a forward-only network? Commented Mar 26, 2018 at 13:36
  • No, it's a normal basic network. I might include some network packets when a signal has to traverse from a processing unit to a "working" unit. I want to make it as modular as possible, so it might, in theory be used over several computers/the internet. Commented Mar 26, 2018 at 14:08

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.