Re: [Servingxml-help] Flat file structure validation
Brought to you by:
danielaparker
Menu
▾
▴
Re: [Servingxml-help] Flat file structure validation
|
From: Ravikumar T. <ra...@gm...> - 2007-05-11 03:56:24
|
yup. That is exactly what we are doing today. On 5/10/07, Daniel Parker <ser...@li...> wrote: > > Sorry for the delayed response. > > Buffering isn't as simple as that. It has to be optional, since some > users are using this tool with very large datasets. Also, note that record > filters can remove some records from a stream, so what was once the previous > record is no longer. > > For now, I would suggest using a custom record filter, do your own > buffering, and modify the current record by adding the fields you need from > previous records. > > Daniel > > ----- Original Message ----- > *From:* Ravikumar Tadysetty <ra...@gm...> > *To:* Daniel Parker <dan...@sy...> > *Sent:* Sunday, May 06, 2007 9:03 PM > *Subject:* Re: [Servingxml-help] Flat file structure validation > > Daniel > > For situations like this one (or more complicated than this), can we do > something like: > > group all records by file number. > Hold all those records in a memory buffer. > Provide a facility to access that memory buffer in the record mapping > section. > eg: > > <group1> > <a> ERecord->[0] </a> > <b> SRecord->[1] </b> > <group1> > <group2> > <c>SRecord->[0]</c> > <d>ERecord->[1]</d> > </group2> > > This way we can access data of any record in that group from any element > of the xml. One of the main constraints today is that servingxml mapping > processes one record at a time. Once that record is processed, it cannot be > accessed further. We donot have the ability to look-ahead or look-behind > while processing a record in a group. > > > On 5/5/07, Ravikumar Tadysetty <ra...@gm...> wrote: > > > > Daniel, Thank you. Really waiting for this. However this was only a part > > of the puzzle. I wanted more flexibility. Please see the below example. I > > added one more field and I cannot achieve this now. Can you suggest > > something for this type of situation. Essentially, while printing <group2> I > > want the data from the previous record. > > > > From the previous example, I added 2 new fields FIELDX, FIELDY. I want > > to put FIELDX from ERecord and FIELD1 from SRecord in <group2>. > > > > FROM > > > > E1001FIELD0FIELDX > > S1001FIELD1FIELD2 > > > > E1002FIELDAFIELDY > > > > S1002FIELDBFIELDC > > > > TO > > > > <ytds> > > <record fileno="1001"> > > <group1> > > <a>FIELD0</a> > > <b>FIELD2</b> > > </group1> > > > > > > <group2> > > <c>FIELD1</c> > > <d>FIELDX</d> > > </group2> > > </record> > > <record fileno="1002"> > > <group1> > > <a>FIELDA</a> > > > > > > <b>FIELDC</b> > > </group1> > > <group2> > > <c>FIELDB</c> > > <d>FIELDY</d> > > </group2> > > </record> > > </ytds> > > > > > > > > On 4/29/07, Daniel Parker <dan...@sy... > wrote: > > > > > > Ravi, > > > > > > This is now supported in version 0.6.6. Refer to the element > > > reference entry for sx:innerGroup, to the last example. > > > > > > -- Daniel > > > > > > ----- Original Message ----- > > > *From:* Ravikumar Tadysetty <ra...@gm...> > > > *To:* Daniel Parker <dan...@sy...> > > > *Sent:* Sunday, March 25, 2007 10:57 PM > > > *Subject:* Re: [Servingxml-help] Flat file structure validation > > > > > > Daniel, > > > > > > sorry for trouble. I am unable to achieve what I wanted to do. I was > > > trying this through the weekend. I am able to achieve only a part of > > > expected output. I am sending my resource script, input file, expected > > > output and actual output. I hope you will help me fix the resource script to > > > get the expected output. I am sending the resource script as attachment. > > > > > > Input - ytds.txt > > > > > > E1002FIELD0 > > > S1002FIELD1FIELD2 > > > > > > ExpectedOutput: > > > > > > <record fileno=1001> > > > <group1> > > > <a>FIELD0</a> > > > <b>FIELD2</b> > > > </group1> > > > <group2> > > > <c>FIELD1</c> > > > </group2> > > > </record> > > > > > > ActualOutput: > > > > > > <record fileno=1001> > > > <group1> > > > <a>FIELD0</a> > > > <b>FIELD2</b> > > > </group1> > > > </record> > > > > > > Regards > > > Ravi > > > > > > On 3/22/07, Daniel Parker <dan...@sy... > wrote: > > > > > > > > Yes, in record mapping sections you can define groupings based on > > > > the previous, current, and next records, check out any of the examples using > > > > sx:innerGroup. The case below is no problem, I may use it in as example in > > > > the element reference. > > > > > > > > Daniel > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > *From:* Ravikumar Tadysetty <ra...@gm...> > > > > *To:* Daniel Parker <dan...@sy...> > > > > *Sent:* Thursday, March 22, 2007 4:06 PM > > > > *Subject:* Re: [Servingxml-help] Flat file structure validation > > > > > > > > Thanks Daniel. This is helpful. > > > > Another question on the same file structure. > > > > > > > > It looks like serving xml processes one record (using onRecord) at a > > > > time. Can I have control over 2 record at a time while converting flat file > > > > to xml. I want to convert the below flat file as follows. Essentially I > > > > want to put one field from E record and one field from S record into > > > > <group1>. So while processing E record, I need information from the next > > > > record also. > > > > > > > > E1001FIELD0 > > > > S1001FIELD1FIELD2 > > > > E1002FIELDA > > > > S1002FIELDBFIELDC > > > > > > > > TO > > > > > > > > <record fileno=1001> > > > > <group1> > > > > <a>FIELD0</a> > > > > <b>FIELD2</b> > > > > </group1> > > > > <group2> > > > > <c>FIELD1</c> > > > > </group2> > > > > </record> > > > > > > > > > > > > > > > > > > > > On 3/12/07, Daniel Parker <dan...@sy...> wrote: > > > > > > > > > > Hello Ravi, > > > > > > > > > > I think you'd have to transform the flat file to XML and express > > > > > the rules in an XML Schema, using the msv:schemaValidator (previously > > > > > msv:msvFilter) to validate the XML. Alternatively, you could write a custom > > > > > record filter in Java, and buffer enough records until you could determine > > > > > whether they were valid, and discard them or throw an exception if you find > > > > > an error. But other than that, the tool doesn't provide anything special > > > > > for validating groups of records, as opposed to individual records. > > > > > > > > > > Best regards, > > > > > Daniel > > > > > > > > > > ----- Original Message ----- > > > > > *From:* Ravikumar Tadysetty <ra...@gm...> > > > > > *To:* ser...@li... > > > > > *Sent:* Friday, March 09, 2007 12:16 AM > > > > > *Subject:* [Servingxml-help] Flat file structure validation > > > > > > > > > > Hi > > > > > > > > > > I am trying to use servingxml to validate and transform flat files > > > > > to xml. I am wondering if this toolkit can help validate the flat file which > > > > > looks as follows: > > > > > > > > > > Every row is a record with position #1 as record type. Position > > > > > #2-5 as filenumber and the rest as misc fields. I want to validate that for > > > > > every filenumber, there exists atleast 1 E record followed by 0 or more S > > > > > records. > > > > > > > > > > E1001FIELD0 > > > > > S1001FIELD1FIELD2 > > > > > S1001FIELD3FIELD4 > > > > > E1002FIELDA > > > > > S1002FIELDBFIELDC > > > > > > > > > > Thanks > > > > > Ravi > > > > > > > > > > ------------------------------ > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > > Join SourceForge.net's Techsay panel and you'll get the chance to > > > > > share your > > > > > opinions on IT & business topics through brief surveys-and earn > > > > > cash > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > > > > > > > > > > > > ------------------------------ > > > > > > > > > > _______________________________________________ > > > > > Servingxml-help mailing list > > > > > Ser...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/servingxml-help > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------- > > > > Take Surveys. Earn Cash. Influence the Future of IT > > > > Join SourceForge.net's Techsay panel and you'll get the chance to > > > > share your > > > > opinions on IT & business topics through brief surveys-and earn cash > > > > > > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > > _______________________________________________ > > > > Servingxml-help mailing list > > > > Ser...@li... > > > > https://lists.sourceforge.net/lists/listinfo/servingxml-help > > > > > > > > > > > > > > |
