@Joliver What if you get 2.5? Will you round it to 2 or to 3?

Alec

That can be also written as

f:=m->(i,j)->Physics:-KroneckerDelta[i mod m, j-1]:
Matrix(3,f(3));

                            [0    1    0]
                            [           ]
                            [0    0    1]
                            [           ]
                            [1    0    0]

Matrix(4,f(4));

                          [0    1    0    0]
                          [                ]
                          [0    0    1    0]
                          [                ]
                          [0    0    0    1]
                          [                ]
                          [1    0    0    0]

_______________
Alec Mihailovs, PhD
Maplesoft Member

That can be also written as

f:=m->(i,j)->Physics:-KroneckerDelta[i mod m, j-1]:
Matrix(3,f(3));

                            [0    1    0]
                            [           ]
                            [0    0    1]
                            [           ]
                            [1    0    0]

Matrix(4,f(4));

                          [0    1    0    0]
                          [                ]
                          [0    0    1    0]
                          [                ]
                          [0    0    0    1]
                          [                ]
                          [1    0    0    0]

_______________
Alec Mihailovs, PhD
Maplesoft Member

@pagan Great testing! It's good to know about such things. At least integer[1] seems to be transferred without problems (which means that inside a byte the order of bits is the same). Could it be that transferring through the network from one computer to another one used something like ntohl and changed the byte order? Was it transferred in the binary mode or in the text mode?

I've just tried ntohl in Windows,

ntohl:=define_external('ntohl',
    'n'::integer[4],
    'RETURN'::integer[4],
    'LIB'="wsock32.dll"):
ntohl(4782969);
                              2046511104
ntohl(%);
                               4782969

It is fast, but not very fast,

A:=Array(1..2^22,4782969,datatype=integer[4]):
time(map[inplace](ntohl,A));
                                5.194
A[1],A[-1];
                        2046511104, 2046511104

Also, I tried RtlUlongByteSwap from ntdll by adding the following to delegates.c,

typedef int (__fastcall *IntFProcInt)(int);

__declspec(dllexport) int _stdcall IntFastInt(
    IntFProcInt myProc, int myInt){
	
	return (myProc)(myInt);		
}

and building the delegates.dll in Visual Studio (Watcom can't deal with __fastcall).

IntFastInt:=define_external('IntFastInt',
    'myProc'::integer[4],
    'myInt'::integer[4],
    'RETURN'::integer[4],
    'LIB'="delegates.dll"):
ntdll:=LoadLibrary("ntdll"):
ByteSwap:=curry(IntFastInt,GetProcAddress(ntdll,"RtlUlongByteSwap")):
ByteSwap(4782969);
                              2046511104
ByteSwap(%);
                               4782969
time(map[inplace](ByteSwap,A));
                                14.024
FreeLibrary(ntdll):

Practically, if one needs that it is better to use bswap, a very simple assembly code (two lines) which could be easily found searching Google, and see my manual,  Writing DLL in Assembler for External Calling in Maple.

Alec

@Axel Vogt It returns a pointer to a structure, which is an integer (the memory address). The same thing works OK in the fopen1 in another thread, and in gzopen with stdcall - they return a pointer to a stream (which is an integer[4] in 32-bit systems and integer[8] in 64-bit). It works OK in Mathematica and Python.

I just tried your example and it worked OK. The handling of integers and floats is different - so it might work for 1 floating input and 1 floating return, but doesn't work with integers.

Alec

@pagan Are you sure? Did you test that? It may happen for floats, because their representation is not standardized. Since that incompatibility is well known, I am pretty sure there are tools converting from one format to another one.

Should be OK for integers though. Headers can be easily added if necessary. The format produced by mentioned Maple commands can be easily reproduced using fread and fwrite. Besides, the same thing can be done with strings, producing text files.

Alec

I just tried the following with 1 variable:

Id:=define_external('gsl_permutation_calloc',
    'n'::integer[4],
    'RETURN'::integer[4],
    'LIB'="libgsl.dll"):
Id(3);

and that crashed Maple. On the other hand, in Mathematica,

In[1]:= Needs["NETLink`"]

In[2]:= Id = 
 DefineDLLFunction["gsl_permutation_calloc", 
  "libgsl.dll", "int", {"int"}, 
  CallingConvention -> "CDecl"];

In[3]:= s = Id[3];

In[4]:= Elem = 
 DefineDLLFunction["gsl_permutation_get", "libgsl.dll", 
  "int", {"int", "int"}, CallingConvention -> "CDecl"];

In[5]:= Table[Elem[s, i], {i, 0, 2}]

Out[5]= {0, 1, 2}

worked as expected. Not even talking about Python, in which things are much more simple,

>>> from ctypes import*
>>> gsl=cdll.libgsl
>>> id=gsl.gsl_permutation_calloc
>>> s=id(3)
>>> elem=gsl.gsl_permutation_get
>>> [elem(s,i) for i in range(3)]
[0, 1, 2]

_______________
Alec Mihailovs, PhD
Maplesoft Member

float[8] arrays could be written and read as simple as binary arrays, just changing integer[1] in fread1 and fwrite1 to float[8] in fread8 and fwrite8, example for 32bit Windows,

fread8:=define_external('fr',
    'ptr'::REF(ARRAY(datatype=float[8])),
    'size'::integer[4],
    'count'::integer[4],
    'stream'::integer[4],
    'RETURN'::integer[4],
    'LIB'="fread.dll"):
fwrite8:=define_external('fw',
    'ptr'::REF(ARRAY(datatype=float[8])),
    'size'::integer[4],
    'count'::integer[4],
    'stream'::integer[4],
    'RETURN'::integer[4],
    'LIB'="fread.dll"):

A:=Array(1..3,[1.25,234.7,0.0018],datatype=float[8]);

    A := [1.25000000000000, 234.700000000000, 0.00180000000000000]

p:=fopen1("fla","wb"):
fwrite8(A,8,3,p):
fclose1(p):
B:=Array(1..4,datatype=float[8]):
p:=fopen1("fla","rb"):
fread8(B,8,3,p):
fclose1(p):
B;

    [1.25000000000000, 234.700000000000, 0.00180000000000000, 0.]

A:=Matrix(2,[1.25,234.7,0.0018, Pi],datatype=float[8]);

                 [ 1.25000000000000      234.700000000000]
            A := [                                       ]
                 [0.00180000000000000    3.14159265358979]

p:=fopen1("mat","wb"):
fwrite8(A,8,4,p):
fclose1(p):
B:=Matrix(2,datatype=float[8]):
p:=fopen1("mat","rb"):
fread8(B,8,4,p):
fclose1(p):
B;

              [ 1.25000000000000      234.700000000000]
              [                                       ]
              [0.00180000000000000    3.14159265358979]

The only problem with that is that the user should take care to avoid the overflowing of B - if more data is written in B than it's size, the Maple kernel vector might get damaged and crash Maple kernel, in the way that the worksheet can't be saved. It is safer to use it not directly, but inside a procedure similar to myreadbytes in the original post, taking care of preventing possible overflowing.

_______________
Alec Mihailovs, PhD
Maplesoft Member

What I mean is that __stdcall is great for GUI - try, for example,

MsgBox:=define_external('MessageBoxA',
    'hWind'::integer[4],
    'lpText'::string,
    'lpCaption'::string,
    'uType'::integer[4],
    'RETURN'::integer[4],
    'LIB'="User32.dll"):
MsgBox(0,"Hello world!","Hello Windows!",0);

Try that with the last argument (or parameter) 1, 2, 3, 4, 5, or 6 instead of 0. All of the functionality of Maplets and much more could be done that way. That's nice!

But not that many people know about that and use it. On the other hand, what many people would like to have is factoring capabilities of PARI with ecm etc., and other such things, and it takes some time to write wrappers. It would be a great improvement if such factoring functions etc. would be accessible as simple as displaying the "Hello world!" message box in Windows as well.

_______________
Alec Mihailovs, PhD
Maplesoft Member

As Acer suggested, I tried fread both in 64bit Linux and in 32bit Windows - the speed is comparable with myreadbytes, and much faster than Maple operates without it.

First, in Linux,

fopen1:=define_external('fopen',
    'filename'::string,
    'mode'::string,
    'RETURN'::integer[8],
    'LIB'="libc.so.6"):
fread1:=define_external('fread',
    'ptr'::REF(ARRAY(datatype=integer[1])),
    'size'::integer[8],
    'count'::integer[8],
    'stream'::integer[8],
    'RETURN'::integer[8],
    'LIB'="libc.so.6"):
fwrite1:=define_external('fwrite',
    'ptr'::REF(ARRAY(datatype=integer[1])),
    'size'::integer[8],
    'count'::integer[8],
    'stream'::integer[8],
    'RETURN'::integer[8],
    'LIB'="libc.so.6"):
fclose1:=define_external('fclose',
    'stream'::integer[8],
    'RETURN'::integer[4],
    'LIB'="libc.so.6"):
A:=Array(1..2^26,97,datatype=integer[1]):
p:=fopen1("aaa","wb"):
time[real](fwrite1(A,1,2^26,p));
                                    0.324
fclose1(p):
p:=fopen1("aaa","rb"):
B:=Array(1..2^26,98,datatype=integer[1]):
time[real](fread1(B,1,2^26,p));
                                    0.135
fclose1(p):
B[1],B[-1];
                                   97, 97
E:=Array(1..2^26,datatype=integer[1]):
time[real](ArrayTools:-Copy(A,E));
                                    0.203

In Windows, first I compiled a dll, I called it fread.dll, using the following code,

#include <stdio.h>

__declspec(dllexport) FILE * __stdcall fo(
    const char * filename, const char * mode ){
	
	return fopen(filename, mode); 
}

__declspec(dllexport) size_t __stdcall fr(
    void * ptr, size_t size, size_t count, FILE * stream ){
	
	return fread(ptr, size, count, stream); 
}

__declspec(dllexport) size_t __stdcall fw(
    const void * ptr, size_t size, size_t count, FILE * stream ){
	
	return fwrite(ptr, size, count, stream); 
}

__declspec(dllexport) int __stdcall fc(
    FILE * stream ){
	
	return fclose(stream); 
}

and then used it in Maple as follows

fopen1:=define_external('fo',
    'filename'::string,
    'mode'::string,
    'RETURN'::integer[4],
    'LIB'="fread.dll"):
fread1:=define_external('fr',
    'ptr'::REF(ARRAY(datatype=integer[1])),
    'size'::integer[4],
    'count'::integer[4],
    'stream'::integer[4],
    'RETURN'::integer[4],
    'LIB'="fread.dll"):
fwrite1:=define_external('fw',
    'ptr'::REF(ARRAY(datatype=integer[1])),
    'size'::integer[4],
    'count'::integer[4],
    'stream'::integer[4],
    'RETURN'::integer[4],
    'LIB'="fread.dll"):
fclose1:=define_external('fc',
    'stream'::integer[4],
    'RETURN'::integer[4],
    'LIB'="fread.dll"):
A:=Array(1..2^26,97,datatype=integer[1]):
p:=fopen1("aaa","wb"):
time[real](fwrite1(A,1,2^26,p));
                                    0.881
fclose1(p):
p:=fopen1("aaa","rb"):
B:=Array(1..2^26,98,datatype=integer[1]):
time[real](fread1(B,1,2^26,p));
                                    0.072
fclose1(p):
B[1],B[-1];
                                   97, 97
E:=Array(1..2^26,datatype=integer[1]):
time[real](ArrayTools:-Copy(A,E));
                                    0.092

_______________
Alec Mihailovs, PhD
Maplesoft Member

Just tried it in the 64bit Linux with Maple 12 and not the newest zlib version 1.2.3.3. and gzread is still much faster,

    |\^/|     Maple 12 (X86 64 LINUX)
._|\|   |/|_. Copyright (c) Maplesoft, a division of Waterloo Maple Inc. 2008
 \  MAPLE  /  All rights reserved. Maple is a trademark of
 <____ ____>  Waterloo Maple Inc.
      |       Type ? for help.
> A:=Array(1..2^26,99,datatype=integer[1]):
memory used=64.9MB, alloc=64.8MB, time=0.19
> time(writebytes("A",A));close("A");
                                     3.170

> B:=Array(1..2^26,1,datatype=integer[1]):
memory used=128.9MB, alloc=128.9MB, time=3.50
> time(readbytes("A",B));close("A");
                                     2.000

> B[1],B[-1];
                                    99, 99

> myreadbytes:=proc(f)
> local gzopen, gzread, gzclose, n, p, A;
> gzopen:=define_external('gzopen',
>     'path'::string,
>     'mode'::string,
>     'RETURN'::integer[8],
>     'LIB'="libz.so.1");
> gzread:=define_external('gzread',
>     'file'::integer[8],
>     'buf'::REF(ARRAY(datatype=integer[1])),    
>     'len'::integer[8],
>     'RETURN'::integer[4],
>     'LIB'="libz.so.1");
> gzclose:=define_external('gzclose',
>     'file'::integer[8],
>     'RETURN'::integer[4],
>     'LIB'="libz.so.1");
> n:=FileTools:-Size(f);
> A:=Array(1..n,datatype=integer[1]);
> try p:=gzopen(f,"rb");
> if gzread(p,A,n)=n
> then return A end if
> finally gzclose(p)
> end try
> end proc:
> time(assign(C=myreadbytes("A")));
memory used=193.0MB, alloc=193.0MB, time=5.57
                                     0.070

> C[1],C[-1];
                                    99, 99

> 'time(myreadbytes("A"))'$5;
memory used=257.7MB, alloc=257.4MB, time=5.63
memory used=321.7MB, alloc=321.5MB, time=5.67
memory used=385.7MB, alloc=385.6MB, time=5.70
memory used=449.7MB, alloc=449.7MB, time=5.73
memory used=513.8MB, alloc=513.9MB, time=5.78
                       0.059, 0.040, 0.030, 0.030, 0.049

> E:=Array(1..2^26,2,datatype=integer[1]):
memory used=577.8MB, alloc=578.0MB, time=5.91
> time(ArrayTools:-Copy(A,E));
                                     0.089

_______________
Alec Mihailovs, PhD
Maplesoft Member

I also think so (buffering). In Windows both fread and read are in the msvcrt dll(s) which have cdecl calling convention, so one can't call them that simple from Maple. It's not that hard to write a wrapper and compile it in a dll with correct calling convention - I think I have an example with mktime somewhere on this site, but it still takes some time. Buffering in zlib is pretty good. The standard distribution, zlib1.dll, also uses cdecl convention, that's one of the reasons I used that one. Plus, this one has some optimized assembly code.

It would be interesting to compare speeds with fread. It uses just a FILE structure (in addition to simple hardware types) so it could be called in Linux using WRAPPER option in define_external, I think.

By the way, Maple 14 includes a version of zlib (called mzlib.dll in Windows), but it has only 2 functions exported there - used in StringTools:-Compress and Uncompress.

_______________
Alec Mihailovs, PhD
Maplesoft Member

By the way, I wrote several functions accessing MapleCloud programmatically from any interface (including Command Line and Classic - I remember the original author of this thread posting about that, but don't have time at the moment to search for that.) That includes getting a list of groups that one can access, the list of posts there, translating their titles, retrieving worksheets in these lists, submitting worksheets (and, perhaps, not only worksheets), chat etc. Unfortunately, only for Maple 14 - I used several functions there (such as Encode(..., percent), Decode(..., percent)) which are not present in Maple 13 or earlier.

It is not finished - not everything is covered - such as approving members in one of your groups, for example - because I didn't have any experience with that, so I couldn't trace the corresponding packets (or messages) - if somebody wants to join test@295, I'll get that, too :).

Also, the login (enabling the access to groups other than public, Maplesoft@admin, and private) is only for Maplesoft accounts - I have a Google email, but didn't try to login to MapleCloud using it.)

If there were a sufficient interest in that I could post what I wrote already (even if it is not finished) on one of my websites - mapleadvisor (named in honor of Robert Israel) perhaps. That also includes some utility functions - such as gzip and gunzip (from Maple - both for files and Arrays) - works faster than standard gzip in Linux or Cygwin, by the way; a proper compression in the zlib format (in Maple 14 Compress in StringTools compresses not only a string, but also a 0 after it - which is not only ridiculous, but also differs from any other zlib implementation that I am aware of), calculations of adler32, crc32 etc.

_______________
Alec Mihailovs, PhD
Maplesoft Member

However, trying to read posts that are not in English is even more strain. Of course, one can always use Google translator - and I can see that the first comment in this thread has title "Communicate in English together, it can get more attention and support" - but it may be more time consuming for people viewing this post (and it may get few thousands views (few hundreds of which are unique) to do that in that direction rather than for the original author to do that in the opposite direction (posting in English.)

For one person for whom you are the world (whether it is you or your wife, or other relative, or maybe you have a stalker - I don't exactly understand what you are trying to say repeating that 10 times or so) it may seem OK to make other people to jump through the hooples to read posts written in other than English languages, and it may be OK for some other people, too, having more free time than me, but I, personally, don't have time for that - and in those few free minutes that I got this Sunday my definite choice is to try to reply to interesting to me posts in English rather than trying to use Google translator again and again to understand what these hieroglyphs mean.

_______________
Alec Mihailovs, PhD
Maplesoft Member

I put that kernelopts(opaquemodules=false) in my Maple ini file a long time ago, together with a series of other useful things - such as setiing the rtablesize in interface to 25, with(plots), setting libname to include a bunch of other libraries etc. (it's rather long) and never had any problems with that as far as I recall. What could be possibly wrong with that?

I think I posted about that usage of :: about a year or so ago on this site, when I discovered it, but it seems as it was unnoticed by most people.

@Joe Riel. Great work with mdb - I'll take a look at the Windows version (sometime soon hopefully). I always enjoy your work!

_______________
Alec Mihailovs, PhD
Maplesoft Member