Hi,
I have a mex-function which reads data from a database several times during a simulation. I read from a previous post that Adigator is not able to work with mex-file, but I have the source C-file and I can rewrite the code in a Matlab function. I was just wondering if Adigator can differentiate this kind of function and so if it worth to convert my C-code in a Matlab version.
Thank you in advance for your help,
Alessandro
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you are just grabbing data (i.e. no inputs or outputs to MEX file contain derivatives) then the best bet is probably to give all of the data as an auxiliary input to your function which you are differentiating. If you cannot load in all of the data due to memory issues then you could save it to one or more .mat files and use the y = load('myfile.mat') inside the file you wish to differentiate - just note that adigator will yell at you if you don't use load in that format.
If you are doing table look-up (interpolation), I have overloaded the MATLAB functions ppval, interp1, and interp2 - interp2 will not accept 'nearest' at this point in time though. If using interp2 you may also want to look into the functions adigatorGenInterp2pp and adigatorEvalInterp2pp. For this type of problem you'd need to get the table data from the C-code and pass it into the function file in one of the two mentioned ways. (another note here is that typically if you are going to repeatedly evaluate the derivative, its best to generate the piecewise polynomials and pass them in rather than the table data itself)
Best,
Matt
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
thank you for your answer and sorry my late reply.
I just converted the C-code into a Matlab function. Actually, the database I'm loading contains the ephemerides of some planets and the function computes the Cartesian position of the desired planet in a certain time, i.e. there is a Newton loop inside my function. I don't know if this kind of loop can be a problem, because I just tried to use Adigator inside GPOPS-II and I have an error.
Sorry, the original question was not clear at all.
Thank you,
Alessandro
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Currently I do not have ADiGator coded to accept while loops, and thus they must be replaced by a for loop with an if statement at the end. Assuming you have some max iteration count on your while loop, then this should be fairly straightforward.
Some other thoughts/suggestions
I am assuming the function is in the continuous file and computing the coordinates at each discrete time point and that time is the only input to this coordinate computation.
ADiGator won't let you loop on the vectorized dimension, so the Newton iterations will need to be vectorized
If initial and final time are fixed, then it would be most efficient to just use the MEX file and write some wrapper files - if this is the case, let me know and I can give a more thorough explanation.
If you have further issues then let me know. Additionally, feel free to attach code here or to email me at weinstein87@gmail.com
-Matt
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ok I wasn't using the function adigatorGenFiles4gpops2.m and I couldn't understand were the problem was. Sorry.
So, now I'm trying to write my code in order to let Adigator differentiate it and I'm having some troubles. So, I have some questions/advices.
1) in adigator.m, at the very beginning of the code, I added the following:
ADIGATOR.ERRORLOCS = [];
in order to initialize that variable and so avoid an error that occurred in the function adigatorError.m
2) I am using the function 'cell' to preallocate some cell variables, but it seems that this function has not been overloaded in cada, am I right? In order to bypass this issue, I just commented the preallocating lines and I added initialization lines, as x = {}.
3) I am having an error given by the use of the function 'mod':
Undefined function 'mod' for input arguments of type 'cada'.
Do I have to solve this problem by using an equivalent of mod, or there is a better way to fix this? I saw that inside the folder @cada there are a number of overloaded functions: those are the only functions I can use?
4) There was a Warning in adigatorTempFiles.m, line 818. I fixed it.
5) There are Warnings when adigator.m/filekeeping (line 691) tries to delete the temporary files stored in the private folder. Why is it trying to delete a file in a private folder? Can Matlab do this?
Thank you very much for your reply.
Alessandro
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The method works by first reading the user's source code and generating what I call an "intermediate source code". In this stage no derivative computations are performed, but rather function procedures are copied over and calls to adigator specific routines are added. These intermediate functions are stored in private folders simply due to matlab's call priority (files contained within private folders are high). I delete old intermediate functions before making new ones as this is the best way to ensure that matlab sees the new ones, (this is the answer to 5). The intermediate source code is then evaluated on overloaded objects, so yes, for each matlab function the user calls, I must also have an overloaded version - these are stored in the @cada directory.
In response to the other items,
This looks like a bug in the adigatorError function when performing my cell/structure pre-allocation run, your fix should be fine.
The cell function is not currently overloaded, the best way to preallocate is to either do
mycell = cell(10,1) -- instantiate using numeric values, not variables
or
mycell{n,1} = []; -- if n is varying this should work
I did not have mod overloaded, see attached for files mod.m, floor.m, cadabinaryarraymath.m
paths are:
adigator/@cada/mod.m
adigator/@cada/floor.m
adigator/@cada/private/cadabinaryarraymath.m
mod calls cadabinaryarraymath, floor is needed for the second derivative as the derivative rule of mod is written in terms of floor
z = mod(x,y) = x - floor(x./y).*y -- the derivative of floor is undefined at integer values but I coded it so that the derivative of floor is always zero, derivative rule for mod reflects this as well.
adigatorPrintTempFiles I assume, could you please provide the fix? I just added this GenErrorLink in and am uncertain what the warning is all about.
What warning is being produced here?
I hope this will help you out and thank you for the interest and the help.
and I just change with the following one only to avoid the Warning:
errlink = sprintf('User''s Function: %s line %1.0d',filename,MajorLineCount,filenameDisp,MajorLineCount);
Now I'm trying to figure out the other issues occur, but I guess most of my issues are due to the fact that my code is written as general as possible in order to allow the use of several phases only by changing the number of phases, stored inside the auxdata structure. This means that the number of phases (i.e. the number of FOR iterations and the length of the variables) depends on an input variable and it is not fixed. Is this an issue for Adigator, right?
In order to try Adigator, I wrote a simplified version of my code, with a single phase problem. Now the generation of the derivatives files runs without problems, but I have an error in the evaluation of my function. May I send to you my code, because now I'm very stuck and I don't know where the error can be.
Thank you very much.
Alessandro
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Feel free to send some code over to weinstein87@gmail.com. For that general code, the number of phases must be fixed at time of adigator file generation so you'll need to generate different derivative files for each case. After you generate them you'll then need a switching statement in your main function to tell gpops which derivative files to use (alternatively just call adigatorGenFiles4gpops2 prior to any gpops2 call).
There have been some issues with conditional statements based off of user flags which switch from one problem to another with a different number of phases - I explain this a little better in the V0.4.2 release notes.
perfect now with your new functions that problem is easily solved! Thank you.
Regarding your point 6, yes the warning was in adigatorPrintTempFiles/GenErrorLink. There was this line
errlink = sprintf(['User''s Function: ',filenameDisp,' line %1.0d'],MajorLineCount,MajorLineCount);
and I just change with the following one only to avoid the Warning:
errlink = sprintf('User''s Function: %s line %1.0d',filename,MajorLineCount,filenameDisp,MajorLineCount);
Now I'm trying to figure out the other issues occur, but I guess most of my issues are due to the fact that my code is written as general as possible in order to allow the use of several phases only by changing the number of phases, stored inside the auxdata structure. This means that the number of phases (i.e. the number of FOR iterations and the length of the variables) depends on an input variable and it is not fixed. Is this an issue for Adigator, right?
In order to try Adigator, I wrote a simplified version of my code, with a single phase problem. Now the generation of the derivatives files runs without problems, but I have an error in the evaluation of my function. May I send to you my code, because now I'm very stuck and I don't know where the error can be.
Thank you very much for your precious help, Matthew.
Actually, I figure out that my 2 main issues were the variable number of phases (I'm trying to solve this problem in the more general way as possible, and your suggestion is very good) and a logical referencing. For this second issue, I just replace the logical indexing with a FOR and IF loops nested. Now it seems working all good for the simple single phase problem. I'm going to write the multiphase and test it.
I really appreciated your help.
Alessandro
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Not a problem and glad to see that it is coming together. For logical referencing/assignment you must use the same logical variable for both the reference(s) and assignment - should be in the user's guide. Using a for/if loop will work too, it just won't be as efficient (also you can't loop over the vectorized dimension in the continuous file).
-Matt
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I have a mex-function which reads data from a database several times during a simulation. I read from a previous post that Adigator is not able to work with mex-file, but I have the source C-file and I can rewrite the code in a Matlab function. I was just wondering if Adigator can differentiate this kind of function and so if it worth to convert my C-code in a Matlab version.
Thank you in advance for your help,
Alessandro
Hello Alessandro,
If you are just grabbing data (i.e. no inputs or outputs to MEX file contain derivatives) then the best bet is probably to give all of the data as an auxiliary input to your function which you are differentiating. If you cannot load in all of the data due to memory issues then you could save it to one or more .mat files and use the y = load('myfile.mat') inside the file you wish to differentiate - just note that adigator will yell at you if you don't use load in that format.
If you are doing table look-up (interpolation), I have overloaded the MATLAB functions ppval, interp1, and interp2 - interp2 will not accept 'nearest' at this point in time though. If using interp2 you may also want to look into the functions adigatorGenInterp2pp and adigatorEvalInterp2pp. For this type of problem you'd need to get the table data from the C-code and pass it into the function file in one of the two mentioned ways. (another note here is that typically if you are going to repeatedly evaluate the derivative, its best to generate the piecewise polynomials and pass them in rather than the table data itself)
Best,
Matt
Hi Matthew,
thank you for your answer and sorry my late reply.
I just converted the C-code into a Matlab function. Actually, the database I'm loading contains the ephemerides of some planets and the function computes the Cartesian position of the desired planet in a certain time, i.e. there is a Newton loop inside my function. I don't know if this kind of loop can be a problem, because I just tried to use Adigator inside GPOPS-II and I have an error.
Sorry, the original question was not clear at all.
Thank you,
Alessandro
Alessandro,
Currently I do not have ADiGator coded to accept while loops, and thus they must be replaced by a for loop with an if statement at the end. Assuming you have some max iteration count on your while loop, then this should be fairly straightforward.
Some other thoughts/suggestions
If you have further issues then let me know. Additionally, feel free to attach code here or to email me at weinstein87@gmail.com
-Matt
Hi Matthew,
ok I wasn't using the function adigatorGenFiles4gpops2.m and I couldn't understand were the problem was. Sorry.
So, now I'm trying to write my code in order to let Adigator differentiate it and I'm having some troubles. So, I have some questions/advices.
1) in adigator.m, at the very beginning of the code, I added the following:
ADIGATOR.ERRORLOCS = [];
in order to initialize that variable and so avoid an error that occurred in the function adigatorError.m
2) I am using the function 'cell' to preallocate some cell variables, but it seems that this function has not been overloaded in cada, am I right? In order to bypass this issue, I just commented the preallocating lines and I added initialization lines, as x = {}.
3) I am having an error given by the use of the function 'mod':
Undefined function 'mod' for input arguments of type 'cada'.
Do I have to solve this problem by using an equivalent of mod, or there is a better way to fix this? I saw that inside the folder @cada there are a number of overloaded functions: those are the only functions I can use?
4) There was a Warning in adigatorTempFiles.m, line 818. I fixed it.
5) There are Warnings when adigator.m/filekeeping (line 691) tries to delete the temporary files stored in the private folder. Why is it trying to delete a file in a private folder? Can Matlab do this?
Thank you very much for your reply.
Alessandro
Alessandro,
The method works by first reading the user's source code and generating what I call an "intermediate source code". In this stage no derivative computations are performed, but rather function procedures are copied over and calls to adigator specific routines are added. These intermediate functions are stored in private folders simply due to matlab's call priority (files contained within private folders are high). I delete old intermediate functions before making new ones as this is the best way to ensure that matlab sees the new ones, (this is the answer to 5). The intermediate source code is then evaluated on overloaded objects, so yes, for each matlab function the user calls, I must also have an overloaded version - these are stored in the @cada directory.
In response to the other items,
This looks like a bug in the adigatorError function when performing my cell/structure pre-allocation run, your fix should be fine.
The cell function is not currently overloaded, the best way to preallocate is to either do
mycell = cell(10,1) -- instantiate using numeric values, not variables
or
mycell{n,1} = []; -- if n is varying this should work
I did not have mod overloaded, see attached for files mod.m, floor.m, cadabinaryarraymath.m
paths are:
adigator/@cada/mod.m
adigator/@cada/floor.m
adigator/@cada/private/cadabinaryarraymath.m
z = mod(x,y) = x - floor(x./y).*y -- the derivative of floor is undefined at integer values but I coded it so that the derivative of floor is always zero, derivative rule for mod reflects this as well.
adigatorPrintTempFiles I assume, could you please provide the fix? I just added this GenErrorLink in and am uncertain what the warning is all about.
What warning is being produced here?
I hope this will help you out and thank you for the interest and the help.
-Matt
mod and floor didn't make it, they are here
Hi Matthew,
perfect now with your new functions that problem is easily solved! Thank you.
Regarding your point 6, yes the warning was in adigatorPrintTempFiles/GenErrorLink. There was this line
errlink = sprintf(['User''s Function: ',filenameDisp,' line %1.0d'],MajorLineCount,MajorLineCount);
and I just change with the following one only to avoid the Warning:
errlink = sprintf('User''s Function: %s line %1.0d',filename,MajorLineCount,filenameDisp,MajorLineCount);
Now I'm trying to figure out the other issues occur, but I guess most of my issues are due to the fact that my code is written as general as possible in order to allow the use of several phases only by changing the number of phases, stored inside the auxdata structure. This means that the number of phases (i.e. the number of FOR iterations and the length of the variables) depends on an input variable and it is not fixed. Is this an issue for Adigator, right?
In order to try Adigator, I wrote a simplified version of my code, with a single phase problem. Now the generation of the derivatives files runs without problems, but I have an error in the evaluation of my function. May I send to you my code, because now I'm very stuck and I don't know where the error can be.
Thank you very much.
Alessandro
Feel free to send some code over to weinstein87@gmail.com. For that general code, the number of phases must be fixed at time of adigator file generation so you'll need to generate different derivative files for each case. After you generate them you'll then need a switching statement in your main function to tell gpops which derivative files to use (alternatively just call adigatorGenFiles4gpops2 prior to any gpops2 call).
There have been some issues with conditional statements based off of user flags which switch from one problem to another with a different number of phases - I explain this a little better in the V0.4.2 release notes.
-Matt
Sent from my iPhone
Thank you very much for your precious help, Matthew.
Actually, I figure out that my 2 main issues were the variable number of phases (I'm trying to solve this problem in the more general way as possible, and your suggestion is very good) and a logical referencing. For this second issue, I just replace the logical indexing with a FOR and IF loops nested. Now it seems working all good for the simple single phase problem. I'm going to write the multiphase and test it.
I really appreciated your help.
Alessandro
Not a problem and glad to see that it is coming together. For logical referencing/assignment you must use the same logical variable for both the reference(s) and assignment - should be in the user's guide. Using a for/if loop will work too, it just won't be as efficient (also you can't loop over the vectorized dimension in the continuous file).
-Matt