From: Heran Q. <hq...@pu...> - 2011-08-15 02:20:16
Attachments:
tora.diff
|
Hi, This is the patch to let tora handle Oracle BLOB properly. Previously tora may either show the string or hex of the content quite randomly. In this patch BLOB will be inserted and updated as stream and the entries in resultdata will always be binary. And the content will always be shown as string. Besides, there is also fixup for my previous patch. Thanks, Heran |
From: Heran Q. <hq...@pu...> - 2011-08-21 23:19:11
|
Any comment? On 08/14/2011 07:19 PM, Heran Quan wrote: > Hi, > > This is the patch to let tora handle Oracle BLOB properly. Previously > tora may either show the string or hex of the content quite randomly. > In this patch BLOB will be inserted and updated as stream and the > entries in resultdata will always be binary. And the content will > always be shown as string. > > Besides, there is also fixup for my previous patch. > > Thanks, > Heran |
From: Nathan N. <nn...@ne...> - 2011-08-22 01:03:39
|
It seems like a great improvement to me, but I'm not fluent enough at that level of the code to know if it's a good patch or not. -- Nathan On 08/21/2011 06:18 PM, Heran Quan wrote: > Any comment? > > On 08/14/2011 07:19 PM, Heran Quan wrote: >> Hi, >> >> This is the patch to let tora handle Oracle BLOB properly. Previously >> tora may either show the string or hex of the content quite randomly. >> In this patch BLOB will be inserted and updated as stream and the >> entries in resultdata will always be binary. And the content will >> always be shown as string. >> >> Besides, there is also fixup for my previous patch. >> >> Thanks, >> Heran > > > ------------------------------------------------------------------------------ > Get a FREE DOWNLOAD! and learn more about uberSVN rich system, > user administration capabilities and model configuration. Take > the hassle out of deploying and managing Subversion and the > tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 > _______________________________________________ > Tora-develop mailing list > Tor...@li... > https://lists.sourceforge.net/lists/listinfo/tora-develop -- ------------------------------------------------------------ Nathan Neulinger nn...@ne... Neulinger Consulting (573) 612-1412 |
From: Mike J. <mrj...@mi...> - 2011-08-22 01:49:45
|
Yikes, sorry for the lack of response to your patch! It looks good to me, let me check it out on some tables and I will commit for you. On Sun, Aug 21, 2011 at 5:40 PM, Nathan Neulinger <nn...@ne...> wrote: > It seems like a great improvement to me, but I'm not fluent enough at that level of the code to know if it's a good > patch or not. > > -- Nathan > > On 08/21/2011 06:18 PM, Heran Quan wrote: >> Any comment? >> >> On 08/14/2011 07:19 PM, Heran Quan wrote: >>> Hi, >>> >>> This is the patch to let tora handle Oracle BLOB properly. Previously >>> tora may either show the string or hex of the content quite randomly. >>> In this patch BLOB will be inserted and updated as stream and the >>> entries in resultdata will always be binary. And the content will >>> always be shown as string. >>> >>> Besides, there is also fixup for my previous patch. >>> >>> Thanks, >>> Heran >> >> >> ------------------------------------------------------------------------------ >> Get a FREE DOWNLOAD! and learn more about uberSVN rich system, >> user administration capabilities and model configuration. Take >> the hassle out of deploying and managing Subversion and the >> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 >> _______________________________________________ >> Tora-develop mailing list >> Tor...@li... >> https://lists.sourceforge.net/lists/listinfo/tora-develop > > -- > ------------------------------------------------------------ > Nathan Neulinger nn...@ne... > Neulinger Consulting (573) 612-1412 > > ------------------------------------------------------------------------------ > uberSVN's rich system and user administration capabilities and model > configuration take the hassle out of deploying and managing Subversion and > the tools developers use with it. Learn more about uberSVN and get a free > download at: http://p.sf.net/sfu/wandisco-dev2dev > _______________________________________________ > Tora-develop mailing list > Tor...@li... > https://lists.sourceforge.net/lists/listinfo/tora-develop > -- Open source monitoring done right: http://www.leemba.com |
From: Mike J. <mrj...@mi...> - 2011-08-22 03:30:29
|
This looks like a good start, but it doesn't work for NCLOB or CLOB types. I think this is because empty_blob() should be empty_clob() or instead. Heran, do you want to take another look at it? Thanks, Mike On Sun, Aug 21, 2011 at 6:19 PM, Mike Johnson <mrj...@mi...> wrote: > Yikes, sorry for the lack of response to your patch! > > It looks good to me, let me check it out on some tables and I will > commit for you. > > > On Sun, Aug 21, 2011 at 5:40 PM, Nathan Neulinger <nn...@ne...> wrote: >> It seems like a great improvement to me, but I'm not fluent enough at that level of the code to know if it's a good >> patch or not. >> >> -- Nathan >> >> On 08/21/2011 06:18 PM, Heran Quan wrote: >>> Any comment? >>> >>> On 08/14/2011 07:19 PM, Heran Quan wrote: >>>> Hi, >>>> >>>> This is the patch to let tora handle Oracle BLOB properly. Previously >>>> tora may either show the string or hex of the content quite randomly. >>>> In this patch BLOB will be inserted and updated as stream and the >>>> entries in resultdata will always be binary. And the content will >>>> always be shown as string. >>>> >>>> Besides, there is also fixup for my previous patch. >>>> >>>> Thanks, >>>> Heran >>> >>> >>> ------------------------------------------------------------------------------ >>> Get a FREE DOWNLOAD! and learn more about uberSVN rich system, >>> user administration capabilities and model configuration. Take >>> the hassle out of deploying and managing Subversion and the >>> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 >>> _______________________________________________ >>> Tora-develop mailing list >>> Tor...@li... >>> https://lists.sourceforge.net/lists/listinfo/tora-develop >> >> -- >> ------------------------------------------------------------ >> Nathan Neulinger nn...@ne... >> Neulinger Consulting (573) 612-1412 >> >> ------------------------------------------------------------------------------ >> uberSVN's rich system and user administration capabilities and model >> configuration take the hassle out of deploying and managing Subversion and >> the tools developers use with it. Learn more about uberSVN and get a free >> download at: http://p.sf.net/sfu/wandisco-dev2dev >> _______________________________________________ >> Tora-develop mailing list >> Tor...@li... >> https://lists.sourceforge.net/lists/listinfo/tora-develop >> > > > |
From: <hq...@pu...> - 2011-08-22 03:39:46
|
OK. I will look into that. -----Original Message----- From: Mike Johnson Sent: Sunday, August 21, 2011 8:30 PM To: Nathan Neulinger Cc: Heran Quan ; tor...@li... Subject: Re: [Tora-develop] BLOB in ORACLE This looks like a good start, but it doesn't work for NCLOB or CLOB types. I think this is because empty_blob() should be empty_clob() or instead. Heran, do you want to take another look at it? Thanks, Mike On Sun, Aug 21, 2011 at 6:19 PM, Mike Johnson <mrj...@mi...> wrote: > Yikes, sorry for the lack of response to your patch! > > It looks good to me, let me check it out on some tables and I will > commit for you. > > > On Sun, Aug 21, 2011 at 5:40 PM, Nathan Neulinger <nn...@ne...> > wrote: >> It seems like a great improvement to me, but I'm not fluent enough at >> that level of the code to know if it's a good >> patch or not. >> >> -- Nathan >> >> On 08/21/2011 06:18 PM, Heran Quan wrote: >>> Any comment? >>> >>> On 08/14/2011 07:19 PM, Heran Quan wrote: >>>> Hi, >>>> >>>> This is the patch to let tora handle Oracle BLOB properly. Previously >>>> tora may either show the string or hex of the content quite randomly. >>>> In this patch BLOB will be inserted and updated as stream and the >>>> entries in resultdata will always be binary. And the content will >>>> always be shown as string. >>>> >>>> Besides, there is also fixup for my previous patch. >>>> >>>> Thanks, >>>> Heran >>> >>> >>> ------------------------------------------------------------------------------ >>> Get a FREE DOWNLOAD! and learn more about uberSVN rich system, >>> user administration capabilities and model configuration. Take >>> the hassle out of deploying and managing Subversion and the >>> tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 >>> _______________________________________________ >>> Tora-develop mailing list >>> Tor...@li... >>> https://lists.sourceforge.net/lists/listinfo/tora-develop >> >> -- >> ------------------------------------------------------------ >> Nathan Neulinger nn...@ne... >> Neulinger Consulting (573) 612-1412 >> >> ------------------------------------------------------------------------------ >> uberSVN's rich system and user administration capabilities and model >> configuration take the hassle out of deploying and managing Subversion >> and >> the tools developers use with it. Learn more about uberSVN and get a free >> download at: http://p.sf.net/sfu/wandisco-dev2dev >> _______________________________________________ >> Tora-develop mailing list >> Tor...@li... >> https://lists.sourceforge.net/lists/listinfo/tora-develop >> > > > |
From: <iv...@cv...> - 2011-08-23 10:16:41
|
Quoting Heran Quan <hq...@pu...>: > Any comment? > > On 08/14/2011 07:19 PM, Heran Quan wrote: >> Hi, >> >> This is the patch to let tora handle Oracle BLOB properly. Previously >> tora may either show the string or hex of the content quite randomly. >> In this patch BLOB will be inserted and updated as stream and the >> entries in resultdata will always be binary. And the content will >> always be shown as string. >> >> Besides, there is also fixup for my previous patch. >> > Oh, I'm sorry for late response - it's summer so please be patient. Rather that commenting your patch I'll explain some patches I made recently. Tora has two different connection providers for Oracle, the "old one" OTL based tooracleconnection.cpp and a new one tooracleconnection_trotl.cpp. The new one is used when you compile Tora with an option -DUSE_TROTL. I implemented this library in order to be able to use more Oracle datatypes and OCI features. These features are not yet used by other tora components, it is to easy to extend toResultModel and be still backward compatible with OTL. For [B|C]LOB manipulation my idea is: - toQValue is a wrapper which can hold any QT datatype. If QT's datatype can not be mapped onto Oracle's type then toQvalue holds an instance of complexType subclass. (Note: toQValue::isBinary returns true for QVariant::ByteArray, which is true for BLOB, RAW, LONG RAW and also for ROWID) - if toQValue::isComplexType returns true then you can get an instance of complexType: toQValue::complexType *i = data.toQVariant().value<toQValue::complexType*>(); - complexType has these interface: virtual bool isBinary() const = 0; virtual bool isLarge() const = 0; // This one return true FOR LOBS virtual QString displayData() const throw() = 0; // Returns subset of data virtual QString editData() const throw() = 0; virtual QString userData() const throw() = 0; virtual QString tooltipData() const throw() = 0; virtual QString dataTypeName() const = 0; virtual QByteArray read(unsigned offset) const = 0; virtual void write(QByteArray const &) = 0; // Repeatable call to read function will return LOB pieces, // until whole LOB is written into a file // write is not used yet. - For far this instances of complexType are returned only by tooracleconnection_trotl.cpp. Other connection providers treat LOBs as QByteArray or as QString. Tora has internal limitation of 20000 Bytes for long string. Any LOB manipulation would lead to a silent LOB truncation, which could be confusing to users. - The problem with LOBs is that they can be very large. And we should not read whole LOB content into memory. Oracle's API for LOBs used so called "LOB Localtors" - something like a filehandle. Oracle Lob have several limitations: - they can not be transferred between sessions - they are opened in read-only mode by default. If you want modify its content you must gain them by using "select from <> for update" or by using "returning clause" in insert/update statement. - On other hand LOB locator are thread-safe and you can spawn or own thread, which will manipulate the LOB, while some other thread will use the same session for other statements. In my opinion any reasonable way of handling LOBs in Tora would require changes in all connection providers code. And this is a point where I got stuck in my work. PS: LOB is not the only large datatype, there are also types like XML, Cursor, collection(table of ...) and LONG. I guess postgresql has even more. Ivan ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
From: Heran Q. <hq...@pu...> - 2011-09-06 03:11:50
Attachments:
tora.diff
|
Hi all, Sorry for my previous patch that I can't fully handle LOB operation. This patch is part of my previous patch but only to fix the problem that once click on any row the status of that row may become "modified" even nothing is done. Thanks, Heran On 08/23/2011 02:35 AM, iv...@cv... wrote: > Quoting Heran Quan <hq...@pu...>: > >> Any comment? >> >> On 08/14/2011 07:19 PM, Heran Quan wrote: >>> Hi, >>> >>> This is the patch to let tora handle Oracle BLOB properly. Previously >>> tora may either show the string or hex of the content quite randomly. >>> In this patch BLOB will be inserted and updated as stream and the >>> entries in resultdata will always be binary. And the content will >>> always be shown as string. >>> >>> Besides, there is also fixup for my previous patch. >>> >> > > Oh, I'm sorry for late response - it's summer so please be patient. > > Rather that commenting your patch I'll explain some patches I made > recently. > Tora has two different connection providers for Oracle, the "old one" OTL > based tooracleconnection.cpp and a new one tooracleconnection_trotl.cpp. > > The new one is used when you compile Tora with an option -DUSE_TROTL. > I implemented this library in order to be able to use more Oracle > datatypes > and OCI features. These features are not yet used by other tora > components, > it is to easy to extend toResultModel and be still backward compatible > with OTL. > > For [B|C]LOB manipulation my idea is: > - toQValue is a wrapper which can hold any QT datatype. > If QT's datatype can not be mapped onto Oracle's type then toQvalue > holds an instance of complexType subclass. > (Note: toQValue::isBinary returns true for QVariant::ByteArray, > which is true > for BLOB, RAW, LONG RAW and also for ROWID) > - if toQValue::isComplexType returns true then you can get an instance > of complexType: > toQValue::complexType *i = > data.toQVariant().value<toQValue::complexType*>(); > > - complexType has these interface: > virtual bool isBinary() const = 0; > virtual bool isLarge() const = 0; // This one return true FOR > LOBS > > virtual QString displayData() const throw() = 0; // Returns > subset of data > virtual QString editData() const throw() = 0; > virtual QString userData() const throw() = 0; > virtual QString tooltipData() const throw() = 0; > > virtual QString dataTypeName() const = 0; > > virtual QByteArray read(unsigned offset) const = 0; > virtual void write(QByteArray const &) = 0; > // Repeatable call to read function will return LOB pieces, > // until whole LOB is written into a file > // write is not used yet. > > - For far this instances of complexType are returned only by > tooracleconnection_trotl.cpp. Other connection providers treat LOBs > as QByteArray or as QString. Tora has internal limitation of 20000 Bytes > for long string. Any LOB manipulation would lead to a silent LOB > truncation, > which could be confusing to users. > > - The problem with LOBs is that they can be very large. And we should not > read whole LOB content into memory. > Oracle's API for LOBs used so called "LOB Localtors" - something like > a filehandle. Oracle Lob have several limitations: > - they can not be transferred between sessions > - they are opened in read-only mode by default. If you want modify its > content you must gain them by using "select from <> for update" > or by using "returning clause" in insert/update statement. > - On other hand LOB locator are thread-safe and you can spawn or own > thread, > which will manipulate the LOB, while some other thread will use > the same > session for other statements. > > In my opinion any reasonable way of handling LOBs in Tora would require > changes in all connection providers code. And this is a point where I > got stuck > in my work. > > PS: LOB is not the only large datatype, there are also types like XML, > Cursor, collection(table of ...) and LONG. I guess postgresql has even > more. > > Ivan > > > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > |
From: Mike J. <mrj...@mi...> - 2011-09-06 17:40:14
|
Hi Heran, Awesome! I'm out at Djangocon this week but if nobody gets to it before me, I'll try to get this committed later tonight. Thanks, Mike On Mon, Sep 5, 2011 at 8:11 PM, Heran Quan <hq...@pu...> wrote: > Hi all, > > Sorry for my previous patch that I can't fully handle LOB operation. This > patch is part of my previous patch but only to fix the problem that once > click on any row the status of that row may become "modified" even nothing > is done. > > Thanks, > Heran > > On 08/23/2011 02:35 AM, iv...@cv... wrote: >> >> Quoting Heran Quan <hq...@pu...>: >> >>> Any comment? >>> >>> On 08/14/2011 07:19 PM, Heran Quan wrote: >>>> >>>> Hi, >>>> >>>> This is the patch to let tora handle Oracle BLOB properly. Previously >>>> tora may either show the string or hex of the content quite randomly. >>>> In this patch BLOB will be inserted and updated as stream and the >>>> entries in resultdata will always be binary. And the content will >>>> always be shown as string. >>>> >>>> Besides, there is also fixup for my previous patch. >>>> >>> >> >> Oh, I'm sorry for late response - it's summer so please be patient. >> >> Rather that commenting your patch I'll explain some patches I made >> recently. >> Tora has two different connection providers for Oracle, the "old one" OTL >> based tooracleconnection.cpp and a new one tooracleconnection_trotl.cpp. >> >> The new one is used when you compile Tora with an option -DUSE_TROTL. >> I implemented this library in order to be able to use more Oracle >> datatypes >> and OCI features. These features are not yet used by other tora >> components, >> it is to easy to extend toResultModel and be still backward compatible >> with OTL. >> >> For [B|C]LOB manipulation my idea is: >> - toQValue is a wrapper which can hold any QT datatype. >> If QT's datatype can not be mapped onto Oracle's type then toQvalue >> holds an instance of complexType subclass. >> (Note: toQValue::isBinary returns true for QVariant::ByteArray, which is >> true >> for BLOB, RAW, LONG RAW and also for ROWID) >> - if toQValue::isComplexType returns true then you can get an instance >> of complexType: >> toQValue::complexType *i = >> data.toQVariant().value<toQValue::complexType*>(); >> >> - complexType has these interface: >> virtual bool isBinary() const = 0; >> virtual bool isLarge() const = 0; // This one return true FOR LOBS >> >> virtual QString displayData() const throw() = 0; // Returns subset >> of data >> virtual QString editData() const throw() = 0; >> virtual QString userData() const throw() = 0; >> virtual QString tooltipData() const throw() = 0; >> >> virtual QString dataTypeName() const = 0; >> >> virtual QByteArray read(unsigned offset) const = 0; >> virtual void write(QByteArray const &) = 0; >> // Repeatable call to read function will return LOB pieces, >> // until whole LOB is written into a file >> // write is not used yet. >> >> - For far this instances of complexType are returned only by >> tooracleconnection_trotl.cpp. Other connection providers treat LOBs >> as QByteArray or as QString. Tora has internal limitation of 20000 Bytes >> for long string. Any LOB manipulation would lead to a silent LOB >> truncation, >> which could be confusing to users. >> >> - The problem with LOBs is that they can be very large. And we should not >> read whole LOB content into memory. >> Oracle's API for LOBs used so called "LOB Localtors" - something like >> a filehandle. Oracle Lob have several limitations: >> - they can not be transferred between sessions >> - they are opened in read-only mode by default. If you want modify its >> content you must gain them by using "select from <> for update" >> or by using "returning clause" in insert/update statement. >> - On other hand LOB locator are thread-safe and you can spawn or own >> thread, >> which will manipulate the LOB, while some other thread will use the >> same >> session for other statements. >> >> In my opinion any reasonable way of handling LOBs in Tora would require >> changes in all connection providers code. And this is a point where I got >> stuck >> in my work. >> >> PS: LOB is not the only large datatype, there are also types like XML, >> Cursor, collection(table of ...) and LONG. I guess postgresql has even more. >> >> Ivan >> >> >> >> >> ---------------------------------------------------------------- >> This message was sent using IMP, the Internet Messaging Program. >> > > > ------------------------------------------------------------------------------ > Special Offer -- Download ArcSight Logger for FREE! > Finally, a world-class log management solution at an even better > price-free! And you'll get a free "Love Thy Logs" t-shirt when you > download Logger. Secure your free ArcSight Logger TODAY! > http://p.sf.net/sfu/arcsisghtdev2dev > _______________________________________________ > Tora-develop mailing list > Tor...@li... > https://lists.sourceforge.net/lists/listinfo/tora-develop > > |