1

My problem is simple, I have 1 computer conected to many powerfull servers. I want to execute the app locally but run the process (heavy load) in the remote servers.

The app+settings vary a lot, and I want that this exactly version of the app+settings folder to be used by the remote instances.

My approach so far:

  • Launch the app locally
  • Use PSEXEC to remote launch the same executable as it is running in local -> in the servers (with a random port number passed by argument)
  • Contect to them via sockets
  • Send commands to execute remotely and get the results

My problem relies in the config files, wich are many(50+) and some of them +4MB. This config files are TXT files in a config folder.

What is the proper way to do it? Is it possible to use PSEXEC to copy remotely a whole folder? Can I do any good trick on the sockets to directly pass a copy of the local files to remote?

I would like all the process to be semi-transparent. Since many people will use it with different versions and settings at the same time. So manually copying the files to 20+servers is NOT an option.

Thank you!

Improve this question
5
  • You might consider using a distributed file system so every server can access the files without copy operations. Also, a job distribution tool such as Gearman might help you make good use of the servers' processing power. Commented Nov 5, 2013 at 15:17
  • If as you say problem is simple - why asking for solution? ;-) Commented Nov 5, 2013 at 15:18
  • 1
    You are already using sockets, why not write the necessary components of the config file to each location as part of the initialization when you are in the process of remote launching the apps. If you are concerned of the size, 4Mb, transfer speeds should make that livable, as long as it is not an ongoing process (i.e. only done during initialization of remote apps) I use something called network variables, but this is a proprietary product from National Instruments and would likely not work for you. Commented Nov 5, 2013 at 15:19
  • @Artur - Simple problem descriptions do not always equate to obvious solution sets. OP is simply asking for advice on his approach. Do you have one? Commented Nov 5, 2013 at 15:21
  • @ryyker I already though that, but it is a looong and hard job :(. I was expecting some trick of PSEXEC to copy a folder or some easy way to link a local file to a remote process via sockets. Commented Nov 5, 2013 at 15:33

2 Answers 2

Reset to default
1

Put the program/script that you want to execute by all machines on one common location on local network (put your configs there too). On all servers create a batch file say 'runme.bat' that will execute your program directly from network location.

This way you can use psexec to run runme.bat essentially executing your program/script on any server you want.

Since often - there are issues using psexec - you may invoke your scripts from Task Scheduler etc.

I do that for 500+ servers and it works. If working for me it will work for you.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks, but that leads to use a network sharing, and manually start/set the config. This needs a lot of manual operation. And I tent to do quick tests with different configs, so it could get repetitive....
No - you put runme.bat on all your hosts just once but change your config + scripts + programs directly from network share as if doing locally.
Ok, that looks interesting. It would require each user to tweak a little bit their config in the local computer. But should be easy to use remotely.
0

You might want to look at HTCondor (http://research.cs.wisc.edu/htcondor/) which could perhaps manage all of this for you.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.