Md5 is an algoritm, that calculates an encodes string of 32 characters out of a given string of theoretically *any* length. The nice thing is, that this md5 key is almost unique, and the second nice thing is, that you can use it as a subscript in globals.
We built a small add-on, zo that this M code could work :
s x="some string, that can be very,very,very large"
d &zmd5(x,.y)
w y
914c6e2ff8ab6ff576f7086121efa50e
My proposal is :
- If anyone wants to be this in GT.M, we could send the source code. (It is not even that hard to make it yourself..). You could do us a faver by putting it into the next release, it spares some time.
- Or is there already a md5 functionality, we missed ?
Greetings,
Alain
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There is not currently any md5 functionality built into GT.M But it is not clear exactly what you want to add to GT.M. A function or something else?
From time to time we do add new $Zxxx() functions, $Zxxx intrisic variables, or new Zxxx commands where the "Z" prefix means they are specific to the GTM implementation. We do this only after careful consideration when either there is no other way to do it or the performance gains are obvious or the usage is so widespread as to be beneficial to everyone. Other extensions to the language are done only in conjunction with changes to the M standard. Our own banking application ships with dozens of external calls to C code that don't warrant inclusion into the M language.
Your md5 code sounds like it would be a handy thing and if there was agreement from a sizeable section of the M community that they would like to see a function of this sort in GT.M I'm sure it could be considered.
By the way, in your example, it looks like you could change this to an external function and call it with $&zmd5(x) and return the value y rather than call it as a routine.
Hope this helps.
Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
If there is consideration of implementing md5 in GT.M, please
please consider building a protocol to the md5 engine so that
one can send a significant dataset to the engine rather than
just the contents of a single string. An example of what I have
in mind is (pseudo M):
set f="filename.ext"
do &md5init() ; initialize the md5 engine
open f use f for read buf quit:buf="" do &md5collect(buf)
close f
set md5=$&md5sum()
This way a large volume of data could be sent in for an
md5sum. As stated, the above is pseudo code... hopefully
the desire is communicated.
Cheers,
--ldl
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I would be already happy, if one could do such a thing like :
s x="some string, that can be very,very,very large"
d &zmd5(x,.y)
And yes, if it is in the function form :
s y=$&zmd5(x)
it would be even better, but I am modest, so this is for me a luxury problem.
ldl, your proposal looks a very nice idea,
but looks to me as a good answer on a somewhat different question .
You could always do a zsy command and fetch the result, and this could be satisfying in some cases, where overhead could not harm. But this is just part 1 of the answer on your question..
My original idea was :
i cannot put very large strings into a global subscript,
or, if it is possible one day, i propably will be punished by a time-consuming $data or $get function.
Hey, we said, let's convert this string by a common algoritm, and you'll always get a short string.
Maybe GT.M could do both ?
Greetings,
Alain
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
What is "very,very,very" long ?? The next production release (V4.4-002) due around the end of the month, while it has some interesting new features, does not have any relief in this area. But there is a field test release also coming out soon based on it that increases the maximum value length for local variables from 32K to 1MB (with a commiserate increase in storage usage) if that helps.
Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We have developed a software for libraries, called 'Brocade', and the only user interface we need is a web browser. We developed it with Cache, but used only the M part of Cache, and actually, our development system works with GT.M. The migration is not finished yet, there are some minor things around, but the engine works fine. It took us about a month, so this seems very short.
One of the things we store in M globals, are the description of books and articles. We often use xml as import/export medium. Some of the softwar is written in Python, where you can put a whole file in one string, if you want. For xml files, this might be interesting to do a read#x Megabyte !.
This limited record and variable length does not compete with other languages.
This is just a suggestion..
Alain
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We have had some discussions about this. For globals, allowing a record to span multiple GDS blocks is a major change to the database engine but for local variables, with the new code, raising the value size limit is currently as easy as modifying a compile time variable (#define) and rebuilding GT.M. We left it at 1M just as a sanity check and to "shake it out" as it were. It wouldn't take much work to turn the limit into a soft limit that is (re)settable by a VIEW command or something similar..
By the way, with the new call-in capabiility of V4.4 that allows C code to call M code, you can write some python C classes that allow you to call your M code directly from python if that is your wish. If I find the time to do this myself I'll post the routines as I'm a python aficionado as well.
Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Not quite at that level of granularity. The call-in functionality allows another language (usually C) to call an M routine . So if you had small M routines that would do the $ORDER or set global values for you, then yes, you could call them in the fashion you describe.
Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Correct. This is not a direct call-in to the lower level GTM functions but a call interface to M routines without having to leave the process. But because of this new support, GTM is now packaged as a shared library so implementation of a DAL interface would be a much easier thing to implement if someone would like to pick that up. Especially since a DAL interface to GTM is defined and exists for VMS.
Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The essential functionality you'll need, is a bidirectional interface to M : python should not only feed M with information, but also get info back.
Maybe by starting an M entry $$Funct^routine(..),
so that the C program could get the return value ?
Or how do you see this ?
One can always put a result in some temp file, and the C program couls read it, but that is not the way we had in mind : let's call this an interface of level 1, level 3 would be better ..
You made me too curious..
Alain
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The callin interface is conceptually similiar to external call interface where M calls C except it is going in the opposite direction. The callin interface has the ability to pass in arguments which can be input, output or both, and get a return value and status code. Arguments can be passed by value or reference.
So there is plenty of functionality to allow python to return information without resorting to a "level 1" interface.. :-)
Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
Md5 is an algoritm, that calculates an encodes string of 32 characters out of a given string of theoretically *any* length. The nice thing is, that this md5 key is almost unique, and the second nice thing is, that you can use it as a subscript in globals.
We built a small add-on, zo that this M code could work :
s x="some string, that can be very,very,very large"
d &zmd5(x,.y)
w y
914c6e2ff8ab6ff576f7086121efa50e
My proposal is :
- If anyone wants to be this in GT.M, we could send the source code. (It is not even that hard to make it yourself..). You could do us a faver by putting it into the next release, it spares some time.
- Or is there already a md5 functionality, we missed ?
Greetings,
Alain
Hi Alain,
There is not currently any md5 functionality built into GT.M But it is not clear exactly what you want to add to GT.M. A function or something else?
From time to time we do add new $Zxxx() functions, $Zxxx intrisic variables, or new Zxxx commands where the "Z" prefix means they are specific to the GTM implementation. We do this only after careful consideration when either there is no other way to do it or the performance gains are obvious or the usage is so widespread as to be beneficial to everyone. Other extensions to the language are done only in conjunction with changes to the M standard. Our own banking application ships with dozens of external calls to C code that don't warrant inclusion into the M language.
Your md5 code sounds like it would be a handy thing and if there was agreement from a sizeable section of the M community that they would like to see a function of this sort in GT.M I'm sure it could be considered.
By the way, in your example, it looks like you could change this to an external function and call it with $&zmd5(x) and return the value y rather than call it as a routine.
Hope this helps.
Steve
Hi,
If there is consideration of implementing md5 in GT.M, please
please consider building a protocol to the md5 engine so that
one can send a significant dataset to the engine rather than
just the contents of a single string. An example of what I have
in mind is (pseudo M):
set f="filename.ext"
do &md5init() ; initialize the md5 engine
open f use f for read buf quit:buf="" do &md5collect(buf)
close f
set md5=$&md5sum()
This way a large volume of data could be sent in for an
md5sum. As stated, the above is pseudo code... hopefully
the desire is communicated.
Cheers,
--ldl
Well,
I would be already happy, if one could do such a thing like :
s x="some string, that can be very,very,very large"
d &zmd5(x,.y)
And yes, if it is in the function form :
s y=$&zmd5(x)
it would be even better, but I am modest, so this is for me a luxury problem.
ldl, your proposal looks a very nice idea,
but looks to me as a good answer on a somewhat different question .
You could always do a zsy command and fetch the result, and this could be satisfying in some cases, where overhead could not harm. But this is just part 1 of the answer on your question..
My original idea was :
i cannot put very large strings into a global subscript,
or, if it is possible one day, i propably will be punished by a time-consuming $data or $get function.
Hey, we said, let's convert this string by a common algoritm, and you'll always get a short string.
Maybe GT.M could do both ?
Greetings,
Alain
What is "very,very,very" long ?? The next production release (V4.4-002) due around the end of the month, while it has some interesting new features, does not have any relief in this area. But there is a field test release also coming out soon based on it that increases the maximum value length for local variables from 32K to 1MB (with a commiserate increase in storage usage) if that helps.
Steve
We have developed a software for libraries, called 'Brocade', and the only user interface we need is a web browser. We developed it with Cache, but used only the M part of Cache, and actually, our development system works with GT.M. The migration is not finished yet, there are some minor things around, but the engine works fine. It took us about a month, so this seems very short.
One of the things we store in M globals, are the description of books and articles. We often use xml as import/export medium. Some of the softwar is written in Python, where you can put a whole file in one string, if you want. For xml files, this might be interesting to do a read#x Megabyte !.
This limited record and variable length does not compete with other languages.
This is just a suggestion..
Alain
We have had some discussions about this. For globals, allowing a record to span multiple GDS blocks is a major change to the database engine but for local variables, with the new code, raising the value size limit is currently as easy as modifying a compile time variable (#define) and rebuilding GT.M. We left it at 1M just as a sanity check and to "shake it out" as it were. It wouldn't take much work to turn the limit into a soft limit that is (re)settable by a VIEW command or something similar..
By the way, with the new call-in capabiility of V4.4 that allows C code to call M code, you can write some python C classes that allow you to call your M code directly from python if that is your wish. If I find the time to do this myself I'll post the routines as I'm a python aficionado as well.
Steve
If this new call-in capability works in both ways, that would mean i could have something like a
def setGlobalReference(globalref, value)
a
def getGlobalreference(value, globalref)
and a
def dollarOrder(globalref)
Or ?
Sounds great . I can't wait..
Alain
Not quite at that level of granularity. The call-in functionality allows another language (usually C) to call an M routine . So if you had small M routines that would do the $ORDER or set global values for you, then yes, you could call them in the fashion you describe.
Steve
So this not the replacement for DAL ?
Alain
Correct. This is not a direct call-in to the lower level GTM functions but a call interface to M routines without having to leave the process. But because of this new support, GTM is now packaged as a shared library so implementation of a DAL interface would be a much easier thing to implement if someone would like to pick that up. Especially since a DAL interface to GTM is defined and exists for VMS.
Steve
The essential functionality you'll need, is a bidirectional interface to M : python should not only feed M with information, but also get info back.
Maybe by starting an M entry $$Funct^routine(..),
so that the C program could get the return value ?
Or how do you see this ?
One can always put a result in some temp file, and the C program couls read it, but that is not the way we had in mind : let's call this an interface of level 1, level 3 would be better ..
You made me too curious..
Alain
The callin interface is conceptually similiar to external call interface where M calls C except it is going in the opposite direction. The callin interface has the ability to pass in arguments which can be input, output or both, and get a return value and status code. Arguments can be passed by value or reference.
So there is plenty of functionality to allow python to return information without resorting to a "level 1" interface.. :-)
Steve