Menu
▾
▴
Re: [Firebird-net-provider] Firebird-net-provider Digest, Vol 139, Issue 4
|
From: LtColRDSChauhan <rds...@gm...> - 2017-11-18 04:15:03
|
On Sat, Nov 18, 2017 at 6:25 AM, < fir...@li...> wrote: > Send Firebird-net-provider mailing list submissions to > fir...@li... > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/firebird-net-provider > or, via email, send a message with subject or body 'help' to > fir...@li... > > You can reach the person managing the list at > fir...@li... > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Firebird-net-provider digest..." > > > Today's Topics: > > 1. Error reading data from the connection Re: > Firebird-net-provider Digest, Vol 139, Issue 3 (LtColRDSChauhan) > 2. Re: Error reading data from the connection Re: > Firebird-net-provider Digest, Vol 139, Issue 3 (LtColRDSChauhan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 17 Nov 2017 19:19:07 +0530 > From: LtColRDSChauhan <rds...@gm...> > To: fir...@li... > Subject: [Firebird-net-provider] Error reading data from the > connection Re: Firebird-net-provider Digest, Vol 139, Issue 3 > Message-ID: > <CAM...@ma...ail. > com> > Content-Type: text/plain; charset="utf-8" > > On Fri, Nov 17, 2017 at 5:38 PM, < > fir...@li...> wrote: > > > Send Firebird-net-provider mailing list submissions to > > fir...@li... > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.sourceforge.net/lists/listinfo/firebird-net- > provider > > or, via email, send a message with subject or body 'help' to > > fir...@li... > > > > You can reach the person managing the list at > > fir...@li... > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of Firebird-net-provider digest..." > > > > > > Today's Topics: > > > > 1. Error reading data from the connection (LtColRDSChauhan) > > 2. Re: Error reading data from the connection (Gerdus van Zyl) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 16 Nov 2017 20:03:05 +0530 > > From: LtColRDSChauhan <rds...@gm...> > > To: fir...@li... > > Subject: [Firebird-net-provider] Error reading data from the > > connection > > Message-ID: > > <CAMG8wxjv-2MEg9j4cxO4bAjXpDGB6-nLVBQiVdSwVr63RqCT8g@mail. > > gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Hello, > > > > 1. I need to frequently copy a table from one database to another. This > has > > started to fail now, apparently due to increase in number of records. > > Attempting to copy about 13,26,000 records fails reporting ?Error reading > > data from the connection?. Number of records that I can still successful > > copy is about 13,24,510. > > > > 3. I copy the in batches of 50,000 records (variable) each. For each > batch, > > i open and close connection. I have tried setting connection pooling true > > and false both. Also varied the batch size to 5000, 100000 etc. > > > > 2. My database and client are on the same machine. Other details: > > > > RAM 16 GB > > > > Client application 64 Bit > > > > OS Windows 10 64 Bit > > > > Firebird ADO.NET Data Provider ? tried both: 5.11.0.0 and 5.12.0 > > > > Firebird 3.0.2.32703_0_Win64 > > > > 3. Please suggest possible solutions. > > > > Regards, > > > > Rajiv > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > > > ------------------------------ > > > > Message: 2 > > Date: Thu, 16 Nov 2017 16:51:38 +0200 > > From: Gerdus van Zyl <ger...@gm...> > > To: "For users and developers of the Firebird .NET providers" > > <fir...@li...> > > Subject: Re: [Firebird-net-provider] Error reading data from the > > connection > > Message-ID: > > <CALvycTnBNwJVQVsgV3tgEkdP-MOwRGCcUJX6G7sF1dqLYmXH3g@ > > mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > You can just catch the exception, rollback the batch transaction if > > possible, reconnect and retry that batch again. > > Having retry logic is the only way to ensure reliability. > > > > I am using try-catch and roll back. > Also retrying again and again after roll back for last 48hrs. > The result is consistently the same ie copying about 13,26,000 records > fails reporting Error reading data from the connection, > while copying about 13,24,510 record always succeeds. > The issue here is more then just ensuring reliability. > May be I am missing out settings. > Please suggest possible solutions. > > > > > On 16 November 2017 at 16:33, LtColRDSChauhan <rds...@gm...> > wrote: > > > > > Hello, > > > > > > 1. I need to frequently copy a table from one database to another. This > > > has started to fail now, apparently due to increase in number of > records. > > > Attempting to copy about 13,26,000 records fails reporting ?Error > reading > > > data from the connection?. Number of records that I can still > successful > > > copy is about 13,24,510. > > > > > > 3. I copy the in batches of 50,000 records (variable) each. For each > > > batch, i open and close connection. I have tried setting connection > > pooling > > > true and false both. Also varied the batch size to 5000, 100000 etc. > > > > > > 2. My database and client are on the same machine. Other details: > > > > > > RAM 16 GB > > > > > > Client application 64 Bit > > > > > > OS Windows 10 64 Bit > > > > > > Firebird ADO.NET Data Provider ? tried both: 5.11.0.0 and 5.12.0 > > > > > > Firebird 3.0.2.32703_0_Win64 > > > > > > 3. Please suggest possible solutions. > > > > > > Regards, > > > > > > Rajiv > > > > > > > > > > > > ------------------------------------------------------------ > > > ------------------ > > > Check out the vibrant tech community on one of the world's most > > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > _______________________________________________ > > > Firebird-net-provider mailing list > > > Fir...@li... > > > https://lists.sourceforge.net/lists/listinfo/firebird-net-provider > > > > > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > > > ------------------------------ > > > > ------------------------------------------------------------ > > ------------------ > > Check out the vibrant tech community on one of the world's most > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > ------------------------------ > > > > Subject: Digest Footer > > > > _______________________________________________ > > Firebird-net-provider mailing list > > Fir...@li... > > https://lists.sourceforge.net/lists/listinfo/firebird-net-provider > > > > > > ------------------------------ > > > > End of Firebird-net-provider Digest, Vol 139, Issue 3 > > ***************************************************** > > > -------------- next part -------------- > An HTML attachment was scrubbed... > > ------------------------------ > > Message: 2 > Date: Sat, 18 Nov 2017 06:25:47 +0530 > From: LtColRDSChauhan <rds...@gm...> > To: fir...@li... > Subject: Re: [Firebird-net-provider] Error reading data from the > connection Re: Firebird-net-provider Digest, Vol 139, Issue 3 > Message-ID: > <CAMG8wxjSMZiWbsWcVxAxdAeqryPNn7R4w+a8JZRpAKx7QsOQ4w@mail. > gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Fri, Nov 17, 2017 at 7:19 PM, LtColRDSChauhan <rds...@gm...> > wrote: > > > > > > > On Fri, Nov 17, 2017 at 5:38 PM, <firebird-net-provider- > > re...@li...> wrote: > > > >> Send Firebird-net-provider mailing list submissions to > >> fir...@li... > >> > >> To subscribe or unsubscribe via the World Wide Web, visit > >> https://lists.sourceforge.net/lists/listinfo/firebird-net-pr > >> ovider > >> or, via email, send a message with subject or body 'help' to > >> fir...@li... > >> > >> You can reach the person managing the list at > >> fir...@li... > >> > >> When replying, please edit your Subject line so it is more specific > >> than "Re: Contents of Firebird-net-provider digest..." > >> > >> > >> Today's Topics: > >> > >> 1. Error reading data from the connection (LtColRDSChauhan) > >> 2. Re: Error reading data from the connection (Gerdus van Zyl) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Thu, 16 Nov 2017 20:03:05 +0530 > >> From: LtColRDSChauhan <rds...@gm...> > >> To: fir...@li... > >> Subject: [Firebird-net-provider] Error reading data from the > >> connection > >> Message-ID: > >> <CAM...@ma... > >> ail.com> > >> Content-Type: text/plain; charset="utf-8" > >> > >> Hello, > >> > >> 1. I need to frequently copy a table from one database to another. This > >> has > >> started to fail now, apparently due to increase in number of records. > >> Attempting to copy about 13,26,000 records fails reporting ?Error > reading > >> data from the connection?. Number of records that I can still successful > >> copy is about 13,24,510. > >> > >> 3. I copy the in batches of 50,000 records (variable) each. For each > >> batch, > >> i open and close connection. I have tried setting connection pooling > true > >> and false both. Also varied the batch size to 5000, 100000 etc. > >> > >> 2. My database and client are on the same machine. Other details: > >> > >> RAM 16 GB > >> > >> Client application 64 Bit > >> > >> OS Windows 10 64 Bit > >> > >> Firebird ADO.NET Data Provider ? tried both: 5.11.0.0 and 5.12.0 > >> > >> Firebird 3.0.2.32703_0_Win64 > >> > >> 3. Please suggest possible solutions. > >> > >> Regards, > >> > >> Rajiv > >> -------------- next part -------------- > >> An HTML attachment was scrubbed... > >> > >> ------------------------------ > >> > >> Message: 2 > >> Date: Thu, 16 Nov 2017 16:51:38 +0200 > >> From: Gerdus van Zyl <ger...@gm...> > >> To: "For users and developers of the Firebird .NET providers" > >> <fir...@li...> > >> Subject: Re: [Firebird-net-provider] Error reading data from the > >> connection > >> Message-ID: > >> <CALvycTnBNwJVQVsgV3tgEkdP-MOwRGCcUJX6G7sF1dqLYmXH3g@mail. > >> gmail.com> > >> Content-Type: text/plain; charset="utf-8" > >> > >> You can just catch the exception, rollback the batch transaction if > >> possible, reconnect and retry that batch again. > >> Having retry logic is the only way to ensure reliability. > >> > > > > I am using try-catch and roll back. > > Also retrying again and again after roll back for last 48hrs. > > The result is consistently the same ie copying about 13,26,000 records > > fails reporting Error reading data from the connection, > > while copying about 13,24,510 record always succeeds. > > The issue here is more then just ensuring reliability. > > > I could pin point the issue in my case. > > With the select command text for FbDataAdapter set as: > > SELECT first 50000 SKIP 1260000 * FROM PATIENT_BILL order by REGDID, SER > > Calling Fill raises the exception Error reading data from the connection > immediately on a freshly created single connection with no pooling. > > Am I missing some setting for FbDataAdapter? > 1. I have tried all combinations of connection string settings for: Connection Timeout, Connection lifetime, Pooling 2. For the DataSet tried FbDataAdapter and FbDataReader both. 3. Still the above issue surfaces consistently. 4. The issue appears to be: By time the records are fetched, connection is broken / closed. 5. Any possible solutions on this? > > > > May be I am missing out settings. > > Please suggest possible solutions. > > > > > > > >> On 16 November 2017 at 16:33, LtColRDSChauhan <rds...@gm...> > wrote: > >> > >> > Hello, > >> > > >> > 1. I need to frequently copy a table from one database to another. > This > >> > has started to fail now, apparently due to increase in number of > >> records. > >> > Attempting to copy about 13,26,000 records fails reporting ?Error > >> reading > >> > data from the connection?. Number of records that I can still > successful > >> > copy is about 13,24,510. > >> > > >> > 3. I copy the in batches of 50,000 records (variable) each. For each > >> > batch, i open and close connection. I have tried setting connection > >> pooling > >> > true and false both. Also varied the batch size to 5000, 100000 etc. > >> > > >> > 2. My database and client are on the same machine. Other details: > >> > > >> > RAM 16 GB > >> > > >> > Client application 64 Bit > >> > > >> > OS Windows 10 64 Bit > >> > > >> > Firebird ADO.NET Data Provider ? tried both: 5.11.0.0 and 5.12.0 > >> > > >> > Firebird 3.0.2.32703_0_Win64 > >> > > >> > 3. Please suggest possible solutions. > >> > > >> > Regards, > >> > > >> > Rajiv > >> > > >> > > >> > > >> > ------------------------------------------------------------ > >> > ------------------ > >> > Check out the vibrant tech community on one of the world's most > >> > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > >> > _______________________________________________ > >> > Firebird-net-provider mailing list > >> > Fir...@li... > >> > https://lists.sourceforge.net/lists/listinfo/firebird-net-provider > >> > > >> > > >> -------------- next part -------------- > >> An HTML attachment was scrubbed... > >> > >> ------------------------------ > >> > >> ------------------------------------------------------------ > >> ------------------ > >> Check out the vibrant tech community on one of the world's most > >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot > >> > >> ------------------------------ > >> > >> Subject: Digest Footer > >> > >> _______________________________________________ > >> Firebird-net-provider mailing list > >> Fir...@li... > >> https://lists.sourceforge.net/lists/listinfo/firebird-net-provider > >> > >> > >> ------------------------------ > >> > >> End of Firebird-net-provider Digest, Vol 139, Issue 3 > >> ***************************************************** > >> > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > > ------------------------------ > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Firebird-net-provider mailing list > Fir...@li... > https://lists.sourceforge.net/lists/listinfo/firebird-net-provider > > > ------------------------------ > > End of Firebird-net-provider Digest, Vol 139, Issue 4 > ***************************************************** > |
