Advanced Search

Question: Parallel tasks writing into multiple files (errors...)

Question:Parallel tasks writing into multiple files (errors...)

FrancescoLR 5 Maple 18
task threads fprintf
February 01 2019
1

Hello,

I am trying to parallelize the output of my Maple18 program, such that it is able to write into multiple smaller output files at the same time, rather than into one unique big file spending much time in this operation...

This is the code:

D_vol[p,m,k,l,cij]:=...; #definition  
T_O_I:=Matrix(...,5);  #matrix of indices

 

new_print_F := proc (start_n, end_n)
local st, en, inter, f90code, fcode, iir, ppr, mmr, kkr, llr, ccijr, codefile, fd;
global T_O_I, D_vol, N, p_gamma; 
st := start_n; en := end_n;
if en-st < 10000 then
f90code := ""; fcode := "";
codefile := cat("D_st_ij_12_vol_P", N, "-Pgamma", p_gamma, "_parall2_", st, ".dat");
fd := fopen(codefile, WRITE, TEXT);   ###### Option A: using fprintf
for iir from st to en do
ppr := T_O_I[iir, 1]; mmr := T_O_I[iir, 2]; kkr := T_O_I[iir, 3]; llr := T_O_I[iir, 4]; ccijr := T_O_I[iir, 5];
#############################
####Option 1: cat###############
##fcode := cat("PDv(", ppr, ",", mmr, ",", kkr, ",", llr, ",", ccijr, ")=", D_vol[ppr, mmr, kkr, llr, ccijr]);
##f90code := cat(f90code, fcode, "\n")
####Option 2: CodeGeneration####
fcode := Fortran(D_vol[ppr, mmr, kkr, llr, ccijr], output = string, limitvariablelength = false, resultname = "t");
fcode := SubstituteAll(fcode, "t", cat("PDv(", ppr, ",", mmr, ",", kkr, ",", llr, ",", ccijr, ")"));
f90code := cat(f90code, fcode)
#############################
end do;
fprintf(fd, %s, f90code); fclose(fd);     ###### Option A: using fprintf
#Text[WriteFile](codefile, f90code);    ###### Option B: using WriteFile
return 1
else
inter := floor((1/2)*en-(1/2)*st)+st;
Start(uscita_print, Task = [new_print_F, st, inter], Task = [new_print_F, inter+1, en])
end if
end proc

 

uscita_print := proc (g, h)
return 1
end proc
 

 

Start(new_print_F, 1, rows)

 

As you can see from the code, I have tried with different combinations of options. The errors that I get are:

-----------------------------------------

(only one file: "rows-1 < 10000")

Option1A = Error, (in fprintf) string expected for string format

Option1B = Error, (in unspecified) Too many levels of recursion for display

Option2A = OK (but there isn't parallelism here....)

Option2B = OK (but there isn't parallelism here....)

--------

(more than one file: "rows-1 > 10000") 

Option1A = Error, (in fprintf) string expected for string format

Option1B = Error, (in unspecified) Too many levels of recursion for display

Option2A = Error, (in StringTools:-Split) first argument must be a string

Option2B = Error, (in StringTools:-Split) first argument must be a string

-----------------------------------------


Instead, if I simplify the definition of D_vol to one single sum (for exemple 2+2), this is what I get:

-----------------------------------------

(only one file: "rows-1 < 10000")

Option1A = OK (but there isn't parallelism here, and the example is trivial 2+2=4)

Option1B = OK (but there isn't parallelism here, and the example is trivial 2+2=4)

Option2A = OK (but there isn't parallelism here, and the example is trivial 2+2=4)

Option2B = OK (but there isn't parallelism here, and the example is trivial 2+2=4)

--------

(more than one file: "rows-1 > 10000") 

Option1A = OK (but the example is trivial 2+2=4)

Option1B = OK (but the example is trivial 2+2=4)

Option2A = Error, (in StringTools:-Split) first argument must be a string

Option2B = Error, (in StringTools:-Split) first argument must be a string

-----------------------------------------

Can you please help me? Any advice wil be really useful....

Thanks
Please Wait...