how to run multiple python scripts at the same time?

i_thamary

I have read some answers to the question here but none of them work for me. I have different training files and it takes three days till it finishes but I can reduce the time if I run these files at the same time. Besides, I need to run the test files directly when I finish the training for all methods.

import os
import subprocess,_multiprocessing
import sys
import gc # Garbage Collector
version = ".".join(map(str, sys.version_info[:3]))
if len(version) >3:
   version=version[:-2];
current_dir = os.path.dirname(os.path.realpath(__file__))
# multiprocessing.Process(["python"+version, os.path.join(current_dir, "main_s_train.py"),os.path.join(current_dir, "SC.py")])
gc.collect()
bots = [subprocess.check_call(["python"+version, os.path.join(current_dir, "main_train.py")]),subprocess.check_call(["python"+version, os.path.join(current_dir, "main_s_train.py")]),subprocess.check_call(["python"+version, os.path.join(current_dir, "SC.py")]),subprocess.check_call(["python"+version, os.path.join(current_dir, "MC.py")])]
modules = map(__import__,bots)
import multiprocessing,subprocess
for bot in (bots):
   p = multiprocessing.Process(target=lambda: __import__(bot))
   p.start()

Is there any suggestion to run multiple python scripts at the same time.

Mark Hebert

For Ubuntu, there is an answer here that may work. You can run multiple programs through bash. The "&" tells it to run the program in the background.

python program1.py &
python program2.py &

Since it sounds like you are using a remote server that has Ubuntu, I would recommend using tmux instead. It allows you to open up multiple sessions, run a program on each on and keep them running after you have closed your connection. It also allows you to enter back into each session, if you need to enter/read anything from your programs. I found this guide helpful when I had to do something similar a few months ago.

You can run batch files on Ubuntu as well. I'm not as familiar with running batch files on Ubuntu, but something like the following should work for you. You can also add while loops, if statements, etc.. Anything you would normally type into the shell can be put into the batch file to automatically run your programs or navigate directories.

#!/bin/bash
ECHO starting training program
# without the "&", it waits for your training program to finish running
python training_program.py
ECHO training program completed

# Adding the "&" tells the programs to run in the background
# You can also use tmux instead, if you want to navigate the different programs
python program1.py &
python program2.py &
ECHO training programs running in the background

The file should be saved with a ".sh" extension, then make the file executable by running the following through your shell.

chmod +x your_batch_file.sh 

If you are using Windows, you could create a batch file that runs all the programs. Here is an example of the file that you can create with your editor of choice:

# If you don't want to see the outputs/print of your training program, 
#     add @ECHO OFF to the start. If you want to see them, remove the @ECHO OFF
@ECHO OFF

# Without "start" before the script to run your training program,
#     the batch file will wait until the training program finishes
python "path\to\your\training_program.py"
ECHO training program completed

# Adding "start" opens it in a new window, and processes the next line
#     without waiting for the program to finish running
start python "path\to\your\program1.py"
ECHO Running program1
start python "path\to\your\program2.py"
ECHO Running program2

# Adding "PAUSE" makes the script wait for you manually type a key to continue,
#     but it is not required. You can add PAUSE anywhere in the script
PAUSE

"start" runs each program in a new window. After you have configured the text, safe the file with a ".bat" extension. Then all you have to do is click on the file to run the batch file, which will open each program in a separate window.

Similarly, you can just run the following from command prompt and it will open them in separate windows as well.

start python path\to\your\program1.py
start python path\to\your\program2.py

But it sounds like you are performing this more than once, in which case a batch file may be more suitable.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

run multiple python scripts at the same time

Multiple PHP scripts run at same time in Cron

How to run multiple python files at the same time?

Run multiple python scripts with arguments in real time

How to run python script for multiple input on multiple cores at the same time?

Is there a way to run multiple bash scripts with one script at the same time?

How to run multiple exe programs at the same time using python?

How to run multiple webdriver python programs at the same time?

How to run multiple python scripts to prometheus

How to run multiple audio files in python scripts

How to run multiple scripts in a python script with args

python run multiple scripts

Is it possible to run multiple asyncio in the same time in python?

Run multiple servers in python at same time (Threading)

Python run multiple functions same time

how to run multiple functions at same time

How to run multiple Laravel projects at same time?

How to run multiple terminal consoles at the same time?

How to run scripts in python?

How to run different python scripts in parallel containers from same image

Run Bash scripts in folder all at the same time

How to run multiple user X sessions on the same computer, at the same time?

How run 2 script python in same time?

how to run two activities at the same time in python

Run multiple python scripts concurrently

How can I run multiple python files at the same time in Mac OSX

How to run multiple Python/Shell scripts from one script

how do I run multiple python scripts simultaniously using QProcess

How to run multiple python scripts from shell one after another

TOP Ranking

  1. 1

    Failed to listen on localhost:8000 (reason: Cannot assign requested address)

  2. 2

    Loopback Error: connect ECONNREFUSED 127.0.0.1:3306 (MAMP)

  3. 3

    How to import an asset in swift using Bundle.main.path() in a react-native native module

  4. 4

    pump.io port in URL

  5. 5

    Compiler error CS0246 (type or namespace not found) on using Ninject in ASP.NET vNext

  6. 6

    BigQuery - concatenate ignoring NULL

  7. 7

    ngClass error (Can't bind ngClass since it isn't a known property of div) in Angular 11.0.3

  8. 8

    ggplotly no applicable method for 'plotly_build' applied to an object of class "NULL" if statements

  9. 9

    Spring Boot JPA PostgreSQL Web App - Internal Authentication Error

  10. 10

    How to remove the extra space from right in a webview?

  11. 11

    java.lang.NullPointerException: Cannot read the array length because "<local3>" is null

  12. 12

    Jquery different data trapped from direct mousedown event and simulation via $(this).trigger('mousedown');

  13. 13

    flutter: dropdown item programmatically unselect problem

  14. 14

    How to use merge windows unallocated space into Ubuntu using GParted?

  15. 15

    Change dd-mm-yyyy date format of dataframe date column to yyyy-mm-dd

  16. 16

    Nuget add packages gives access denied errors

  17. 17

    Svchost high CPU from Microsoft.BingWeather app errors

  18. 18

    Can't pre-populate phone number and message body in SMS link on iPhones when SMS app is not running in the background

  19. 19

    12.04.3--- Dconf Editor won't show com>canonical>unity option

  20. 20

    Any way to remove trailing whitespace *FOR EDITED* lines in Eclipse [for Java]?

  21. 21

    maven-jaxb2-plugin cannot generate classes due to two declarations cause a collision in ObjectFactory class

HotTag

Archive