Activity for YADE (SOSFTP)
-
Jairo Fernandez posted a comment on discussion Help
Hi folks, I have JS7 2.5.1 deployed on a kubernetes cluster. From JOC When I try to click on File Transfer option, I got an error saying "A doctype was passed into the xml document" I'm attaching the JOC log file with the complete trace of the error. Any help would be appreciated. BR Jairo
-
Andreas modified ticket #4
Change timezone in Yade and job Scheduler
-
Hi Askhat, for branch 1.x of JobScheduler you create orders from files in xml format (that alternatively can be created from the JOC Cockpit GUI with versions > 1.13.3). The xml format of the order contains for example the following elements: <order> <params/> <script/> <run_time time_zone="Etc/UTC" single_start="08:00:00"/> </order> Timezone identifiers are specified in the tz database. In the above example the start time 8am is considered in the UTC time zone. If no time zone is specified then...
-
Askhat created ticket #4
Change timezone in Yade and job Scheduler
-
Matt posted a comment on discussion Help
More tests: Same issue with yade 1.13.21 (smbj) Regards Matt
-
Hello, I maked more tests with yade 1.13.20 and smbj (only the rename was impacted) /<share>/<directory>/ == KO /<share>/<directory> == KO <share> == OK Regard Matthias
-
My previous post confirms that we are actively adopting new jcifs-ng releases. The YADE will include both smbj and jcifs-ng for the next future. The recent change to make smbj the default SBM provider will remain in place as long as we do not find major problems with smbj. As of now smbj is the more active project which suggests more timely delivery of fixes and improvements (SMB3 offers a number of extensions not yet covered by both smbj and jcifs-ng).
-
perhaps my mistake was relative to the first "/" not defined, I'll make more tests asap :)
-
I have tested last week jcifs with the variable JS7_VFS_SMB_PROVIDER=JCIFS. But I get a class cast exeption with the yade client included with JS7 agent, So I stayed on the regular smbj framework. Do you know if YADE in the futur, will rollback to jcifs or continue to have the both ?
-
Let me summarize the recommended YADE configuration:; Host: Hostname or IP address. This should not include the name of the share. Directory: /<share>/<directory>/<sub-directory> The leading / is required, followed by the name of the share and any directories and sub-directories. Forward slashes / are used to separate directories for Unix and Windows platforms. Use of backslashes is not recommended though this might work.
-
Hi Matt, thank you, I will check with developers. As your file share offers SMB2 you have the option to fallback to use of jcifs-ng instead of smbj. With YADE-591: Switch SMB data provider from jcifs-ng to smbj you have the option to use the command line argument -smb_provider=JCIFS or the environment variable JS7_VFS_SMB_PROVIDER=JCIFS to fallback to use of jcifs-ng. The upcoming YADE release 1.13.21 (next week) will bring YADE-615: SMB - upgrade jcifs-ng data provider to version 2.1.10 which confirms...
-
Andreas, I maked more tests: And I have a workaround. If I define the root of the smb share to the target directory, I 'm able to not use "\" or "/" in my definition. And in this case, it's working great. Regards Matt
-
Hello Andreas, :) I already make the tests, without success :( dir: pricat\test_matthias\ filename: PRICAT_plop.DAT 2023-12-05T08:48:27,890 INFO main c.h.s.s.Session - Connecting to \\10.1.110.39\pricat on session 835642393100397 2023-12-05T08:48:27,938 INFO main c.s.D.SOSDataExchangeEngine - [source][pricat/test_matthias/][recursive=false][fileSpec=PRICAT_plop.DAT][0.011436s]1 files found 2023-12-05 08:48:28.100+01:00 [STDERR] 2023-12-05T08:48:28,008 ERROR main c.s.v.c.SOSFileListEntry - [1][transfer_aborted][Source=pricat/test_matthias/PRICAT_plop.DAT][smb][x.x.x.x][svc_caterflux@x.x.x.x:445][rename][pricat/test_matthias/PRICAT_plop.DAT][pricat/test_matthias/202312050848_PRICAT_plop.DAT.done]com.hierynomus.mssmb2.SMBApiException:...
-
Hi Matt, please try using backslashes instead of forward slashes when using a Windows file share.
-
\o/ youhouuuuuuu Many thanks Andreas, I'll try today There is a paypal for pay a beer anywhere ? Regards Matt
-
Ouch, The previous issue is not appear anymore, but I get a new one: 2023-12-04T14:36:28,502 ERROR main c.s.v.c.SOSFileListEntry - [1][transfer_aborted][Source=pricat/test_matthias/PRICAT_plop.DAT][smb][x.x.x.x][y.y.y.y@z.z.z.z:445][rename][pricat/test_matthias/PRICAT_plop.DAT][pricat/test_matthias/202312041436_PRICAT_plop.DAT.done]com.hierynomus.mssmb2.SMBApiException: STATUS_OBJECT_NAME_INVALID (0xc0000033): SetInfo failed for SMB2FileId{persistentHandle=15 15 60 00 be 00 00 00} (STATUS_OBJECT_NAME_INVALID...
-
Hi Matt, thank you, see https://en.m.wikipedia.org/wiki/File:Charlottenburg_Giesebrechtstra%C3%9Fe_15.JPG :from the company office in the top floor there is a short way down to the bar. You're welcome to make a visit when you're in Berlin. Best regards Andreas
-
\o/ youhouuuuuuu Many thanks Andreas, I'll try today There is a paypal for pay a bear anywhere ? Regards Matt
-
Hi Matt, we are aware of this, see YADE-612: SMB - rename operation The fix is available from YADE 1.13.20. Your JS7 2.6.2 includes YADE 1.13.19. You can download the YADE Standalone Client 1.13.20 from https://www.sos-berlin.com/en/yade-downloads. Extract the Client tarball. In the Agent's installation directory rename the existing "yade" sub-directory and create a "yade" directory from the extracted tarball. Please let us know if this resolves the issue. Best regards Andreas
-
Hello Andreas :) Many thanks for your time, your always a good help :) I use JS7 2.6.2 (with yade inside JS7 2.6.2 agent). Regards Matt
-
I can find this issue on smbj https://github.com/hierynomus/smbj/issues/355, That's seem to be the exact problem. A bad smbj integration inside yade ? Do you think it's possible to fix this one ? Regards Matt
-
Hi Matt, it will be helpful to understand which YADE version you are using. The YADE ships as a standalone client or is bundled with JS1 / JS7. Current JS7 releases 2.5.6 and 2.6.3 include YADE 1.13.20 for which the smbj library has been updated to smbj-0.12.2. Best regards Andreas
-
Hello, If I try to copy file and rename on the source I get this issue: 2023-12-01 13:13:31.603+01:00 [STDERR] 2023-12-01T13:13:31,492 ERROR main c.s.v.c.SOSFileListEntry - SOSVfs_E_229: error. unable to transfer data, reason: com.sos.JSHelper.Exceptions.JobSchedulerException: SOSVfs_E_0150: getreplacing().doReplace aborted with an exception; ([smb][x.x.x.x][administrator@x.x.x.x:445][rename][Repository/Test/test_matt/PRICAT_plop.DAT][Repository/Test/test_matt/202312011313_PRICAT_plop.DAT.done] (STATUS_ACCESS_DENIED...
-
Hello, Anyone have an idea ? :) For information, it's working great if I use linux samba, only windows share seems to be impacted. Regards Matthias
-
Hello, If I try to copy file and rename on the source I get this issue: 2023-12-01 13:13:31.603+01:00 [STDERR] 2023-12-01T13:13:31,492 ERROR main c.s.v.c.SOSFileListEntry - SOSVfs_E_229: error. unable to transfer data, reason: com.sos.JSHelper.Exceptions.JobSchedulerException: SOSVfs_E_0150: getreplacing().doReplace aborted with an exception; ([smb][x.x.x.x][administrator@x.x.x.x:445][rename][Repository/Test/test_matt/PRICAT_plop.DAT][Repository/Test/test_matt/202312011313_PRICAT_plop.DAT.done] (STATUS_ACCESS_DENIED...
-
KayKu posted a comment on discussion Help
Hi team, I have seen several post asking about calculate yesterday date but no one has a solution about. Is possible to pass a custom argument / parameter to a YADE job and use it in filename¿?
-
Glenn Strouhal posted a comment on discussion Help
Also I suspect this is related but the ini to xml conversion utility added the following to the new XML file: <NotificationFragments> <BackgroundServiceFragment name="BackgroundServiceFragment:0"> <BackgroundServiceHost><![CDATA[localhost]]></BackgroundServiceHost> <BackgroundServicePort><![CDATA[4444]]></BackgroundServicePort> <BackgroundServiceJobChainName><![CDATA[/sos/jade/jade_history]]></BackgroundServiceJobChainName> </BackgroundServiceFragment> </NotificationFragments> And in individual profiles,...
-
This is probably an easy, maybe even stupid, question but after upgrading from 1.11.6 to 1.13.11, I can't seem to get what was formerly called a HistoryFileName. I was using INI before. This is what I added to the XML file. Is this not the new equivalent? <General> <Logging> <LogFile><![CDATA[${HOME}/logs/yade_history.csv]]></LogFile> </Logging>
-
chawki kheddam posted a comment on discussion Help
Hello, Since new versions of Yade I guess 1.12.9, command line options doesn't behave as expected. example -target_host doesn't operate in yade 1.13.11. Does Yade not support command line options any more ? Thank you Best regards
-
YADE (SOSFTP) released /YADE.1.13/YADE.1.13.10/jade_client_linux.1.13.10.tar.gz
-
-
-
-
Bernard Jaulin created ticket #5
ERROR StatusLogger Reconfiguration failed: No configuration found for '377dca04'
-
Incorrect link for documentation (yade or jade.sh)
-
Thank you so much. I got the updated file. You should know that the default Download button points to sosftp_client.0.9.1.tar.gz!
-
Hi Glenn, we provided YADE 1.13.10 for download from SourceForge and from our web site that includes Log4j version 2 and respective fixes for CVE-2021-44228 (JOC-1184) and CVE-2021-45046 (JOC-1186) CVE-2021-45105 is not considered relevant for JobScheduler/YADE. Best regards Andreas
-
Solved. That log4j bridge does not translate everything to 2.X. That special renderer is an example of something that log4j-1.2-api-2.17.0.jar does not handle so if anyone else encounters this know that you are just out of luck with a drop in fix.
-
Anyone still watching this forum? We are currently running 1.11.6 on Linux and I am trying to migrate from the installed log4j-1.2.16.jar by using the log4j 1.x bridge. Unsuccessfull so far. I've different tried different variations of jar files but they all give the same error. Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/or/RendererMap at org.apache.log4j.Hierarchy.<init>(Hierarchy.java:97) at org.apache.log4j.LogManager.<clinit>(LogManager.java:82) at org.apache.log4j.Logger.getLogger(Logger.java:117)...
-
-
-
-
-
-
-
-
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.9
-
-
-
Vinutha Yenagi created ticket #3
Credential Store not working with Yade on linux
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.8
-
-
-
ss1234 posted a comment on discussion Help
The result of running above INI is that it shows 3 files transferred, but files get overwritten instead of appended. So finally only 1 file content is present in mergedFile.txt
-
Hello, I am trying to append the content of multiple files using an INI triggered by jade.sh I have multiple files in my source directory like: 1File.txt 2File.txt 3File.txt I need these files to be transferred to target directory with name MergedFile.txt having content of all the 3 files appended one after the other. I have tried the append_files parameter but that did not work. Below is the INI: [jadeHistory] log_filename=/app/jade/logs/jade_history_[date:yyyyMMddHHmmSS].log HistoryFileName=/app/jade/logs/jade_history_[date:yyyyMMddHHmmSS].csv...
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.7
-
-
-
Jack Gross posted a comment on discussion Open Discussion
We have recently deploy SOS-Berlin and we are using YADE to manage file transfers. As we do not want to store credentials in job xml we have decided to use a Credential Store. Looking at Keypass there are three options to control access to the database: Password protected Secure Key Windows Authentication Option 1 is not acceptable as it would require that the password to the keypass database would be stored in the job xml. In fact, because we are running in a Windows environment option 3 would be...
-
Gabriel modified a comment on discussion Help
I was using YADE versoin 1.13.6 and created a simple ftp test profile to retrieve a list of files from a direcory. When I test is on a directory with white space, example "test dir", it dosen't return any file listing. However, when I removed the white space, "test_dir", it works just fine. Is this a possible bug? . This is the profile I was testing. [test] ;; This is comment file_spec = .* operation = getlist source_host = some host source_protocol = ftp source_port = 21 source_user = some user...
-
-
I've create a test profile and I am trying to perform a simple ftp getlist operation from a directory which has a white space in the name. Example of the profile is listed below. [test] ;; This is comment file_spec = .* operation = getlist source_host = some host source_protocol = ftp source_port = 21 source_user = some user source_password = XXXX source_dir = /test dir When I ran the jade.sh script, it just show no files found even though I can see the files sitting in the direcory. However, when...
-
Spencer Shiley modified a comment on discussion Help
I'm trying to see if JobScheduler might be a good fit for our company, but there are several requirements that I'm not sure it will do for us. We're trying to send files to a partner for processing, but at the start of the session we need to set parameters on their end so they know how to handle the incoming files. So in a 'typical' SFTP session we have to pull these 'files': get /command/standard=NONE get /command/parm=SA=SENFDGXS_FTP;RA=SENFDGXS;DOC=O004 before we can do our mput. Digging through...
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.6
-
-
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.5
-
-
-
I'm trying to see if JobScheduler might be a good fit for our company, but there are several requirements that I'm not sure it will do for us. We're trying to send files to a partner for processing, but at the start of the session we need to set parameters on their end so they know how to handle the incoming files. So in a 'typical' SFTP session we have to pull these 'files': get /command/standard=NONE get /command/parm=SA=SENFDGXS_FTP;RA=SENFDGXS;DOC=O004 before we can do our mput. Digging through...
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.4
-
-
-
Hi Madhur, when looking for help it is a good idea to check for the correct forum. This discussion forum is for the YADE (SOSFTP) whereas your question is about JobScheduler. Find an answer to your post from https://sourceforge.net/p/jobscheduler/discussion/486122/thread/44c192fa3e/ Best regards Andreas
-
Madhur posted a comment on discussion Help
Hi, I am using SOS REST API functionality to extract the information of job status. I have attached my code herewith. I am able to perform login and logout functionality also I could test few xmls like show status. terminate scheduler etc. I am stuck show job xmls and unable to get the status of job or job chain. It says error as bad request. Could you please advise what should be correct synatx? Thanks, Madhur
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.3
-
-
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.2
-
-
-
JobScheduler, JOC Cockpit & YADE maintenance release 1.13.1
-
-
-
Maik Opitz posted a comment on discussion Help
Hi, I try to create a job which deletes files on FTP. But it should only delete files older than one day. How can I do this? I tried FileAge = 1d and min_file_age = 24:00 but this did not work. The Job deleted all files. I have two option to achieve this. First is the real file date. So when it was uploaded. The second is the filename which also has a date in it. In the documentation I found an option how to select files with the current date in the filename. But I need the opposite: File which has...
-
-
-
Arthur de Araujo posted a comment on discussion Help
Hi, we are using jobscheduler with Yade for a shor time now. We have a different network where we have multiple agents making transfer files with .ini to make it. But there an issue , this method is not logging 'AS' a transfer but like a simple tasks . How we can log the transfer file ? Regards, Arthur.
-
brasil borba posted a comment on discussion Help
Hi, I have several atomic job chains which can operate independently of each other. The scenario is that the first time they execute a predecessor chain must have already been executed. Think like this: (a) load all PERSON related entities (b) load all ACCOUNT related entities See that account needs person to be loaded. If person has already been loaded once, I can make change to code and reload account at any time, meaning the chain can be triggered standalone. For the first load I have a "trigger"...
-
Magnus Lubeck posted a comment on discussion Help
We got rid of the thousands of files in the source directory, but I weirdly see plenty of waits and a lot of file transfers hanging. Will dig deeper into this. ps -ef | grep java ops 27726 27709 0 15:27 ? 00:00:02 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Dlog4j.configuration=file:///var/lib/ops/mft/app/bin/log4j.properties -classpath /var/lib/ops/mft/app/lib/3rd-party/*:/var/lib/ops/mft/app/lib/sos/* com.sos.DataExchange.SOSDataExchangeEngineMain -settings=/tmp/ft.INFRA-1168.WgSgTm -profile=INFRA-1168-dest...
-
Hi, In our environment, we have ca 30 different file transfer contracts, which each have their own configuration file (.ini file). One of the file transfers (we call it INFRA-841) have thousands of files in the source directory (we rename transferred files to <filename>.done, and only include files without that extension. Our file provider removes the .done-files peridically. Due to the large number of files, YADE takes a very long time to compile the file listing (sometimes up to 10 minutes), which...
-
Thanks a lot for the pointers! I had been looking at https://kb.sos-berlin.com/display/PKB/How+to+set-up+the+Credential+Store, which made me believe that by pointing target_CredentialStore_KeyPath to the entry in my keepass database, it would automatically use the username and password. I got it to work both with a keepass2 kdbx file, and a keepassa file by adding the target_user and target_password clauses that Andreas mentioned above. Probably there is a difference there between 1.11 and 1.12,...
-
Magnus, Here is part of my 1.11 config. Be sure you are using the older keepass database. [globals] include = MailSettings , JADEHistory, BackgroundService title = Global Profile thishost = ${LOCALIP} basedir = ${HOME} serverenv = ${SERVERENV} yadeenv1 = ${YADEENV1} yadeenv2 = ${YADEENV2} targetenv = ${TARGETENV} kpdb = ${KPDB} ;; ;; Default Transfer Settings ;If you transfer files via SFTP then we recommend the following option to increase the transfer speed: load_class_name = com.sos.VirtualFileSystem.SFTP.SOSVfsSFtpJCraft...
-
Hi Magnus, the title of your post is wrong: you are using a KeePass 1 database (.kdb), not KeePass 2 (.kdbx). However, this does not make any difference for use of a credential store with YADE. Your configuration is missing the specification of the target_user and target_password, see YADE Parameter Reference - SMBAuthentication. You have to specify both parameters and assign the path to the entry in the KeePass database, e.g. like this: target_user = cs://some/path/entry_title@user target_password...
-
I am trying to use a credential store, but just does not seem to be able to set the target_user and target_password. It feels like YADE ignores my target_CredentialStore_KeyPath configuration parameter. Anyone else that have ran into this issue? What am I doing wrong? Version = 1.12.8 (2018-12-13 16:27, revision 4ec4e5d4eecbbd845d0c8d4cc649a87abe25c692) Copyright 2003-2018 SOS GmbH Berlin OS: Ubunu 18.04.1 LTS Pre recs sudo apt install kpcli I have created a simple keepass2 file etc/mft.kdb ops@com-fs-l01:/var/lib/ops/mft$...
-
I am trying to use a credential store, but just does not seem to be able to set the target_user and target_password. It feels like YADE ignores my target_CredentialStore_KeyPath configuration parameter. Anyone else that have ran into this issue? What am I doing wrong? Version = 1.12.8 (2018-12-13 16:27, revision 4ec4e5d4eecbbd845d0c8d4cc649a87abe25c692) Copyright 2003-2018 SOS GmbH Berlin OS: Ubunu 18.04.1 LTS Pre recs sudo apt install kpcli I have created a simple keepass2 file etc/mft.kdb ops@com-fs-l01:/var/lib/ops/mft$...
-
Hi, I am experiencing a similar issue. sftp Remote server has a directory with up to 15'000 files, that might or might not match my search criteria (fetched files are renamed to FILENAME.done after transfer) The file transfers start ca 10 - 15 minutes after jade has been started My gut feeling telle me that there is something that jade does after getting the file listing (sftp file listing takes a couple of seconds). I have not yet dug into this in depth.
-
1 >