#1842 linein performance issue

[Erich]

On Windows reading the first line with linein has a runtime that goes up with the square of the line length, e. g.
10mb 3.7 s
100mb 337.6 s
Our variable line read code extends the read buffer by a small fixed amount each time more space is required which a seemingly inefficient realloc doesn't handle well.

Doubling the read buffer size when more space is needed should fix this (gaining orders of magnitudes for e. g. a one GB linein).

This seems to be a lesser issue on Linux, but the proposed fix will still provide some speed-up.

[Erich]

Committed code fix with revision [r12511].

[Rick McGuire]

On Fri, May 12, 2023 at 2:56 PM Per Olov Jonsson perolovjonsson@users.sourceforge.net wrote:

• Group: 5.0.0 --> 5.1.0
  

In general, the Group is usually left as "none" until there is a fix
available and it decided when it will be available.

---

[bugs:#1842] linein performance issue

Status: accepted
Group: 5.1.0
Created: Sun Sep 11, 2022 03:56 PM UTC by Erich
Last Updated: Tue Dec 20, 2022 12:09 PM UTC
Owner: Erich

On Windows reading the first line with linein has a runtime that goes up
with the square of the line length, e. g.
10mb 3.7 s
100mb 337.6 s
Our variable line read code extends the read buffer by a small fixed
amount each time more space is required which a seemingly inefficient
realloc doesn't handle well.

Doubling the read buffer size when more space is needed should fix this
(gaining orders of magnitudes for e. g. a one GB linein).

This seems to be a lesser issue on Linux, but the proposed fix will still
provide some speed-up.

---

Sent from sourceforge.net because you indicated interest in <
https://sourceforge.net/p/oorexx/bugs/1842/>

To unsubscribe from further messages, please visit <
https://sourceforge.net/auth/subscriptions/>

[Per Olov Jonsson]

Point taken

Hälsningar/Regards/Grüsse,
P.O. Jonsson
oorexx@jonases.se

On 12. May 2023, at 20:58, Rick McGuire bigrixx@users.sourceforge.net wrote:

On Fri, May 12, 2023 at 2:56 PM Per Olov Jonsson perolovjonsson@users.sourceforge.net wrote:

• Group: 5.0.0 --> 5.1.0
  

In general, the Group is usually left as "none" until there is a fix
available and it decided when it will be available.

---

[bugs:#1842] linein performance issue

Status: accepted
Group: 5.1.0
Created: Sun Sep 11, 2022 03:56 PM UTC by Erich
Last Updated: Tue Dec 20, 2022 12:09 PM UTC
Owner: Erich

On Windows reading the first line with linein has a runtime that goes up
with the square of the line length, e. g.
10mb 3.7 s
100mb 337.6 s
Our variable line read code extends the read buffer by a small fixed
amount each time more space is required which a seemingly inefficient
realloc doesn't handle well.

Doubling the read buffer size when more space is needed should fix this
(gaining orders of magnitudes for e. g. a one GB linein).

This seems to be a lesser issue on Linux, but the proposed fix will still
provide some speed-up.

---

Sent from sourceforge.net because you indicated interest in <
https://sourceforge.net/p/oorexx/bugs/1842/>

To unsubscribe from further messages, please visit <
https://sourceforge.net/auth/subscriptions/>

---

[bugs:#1842] linein performance issue

Status: accepted
Group: 5.1.0
Created: Sun Sep 11, 2022 03:56 PM UTC by Erich
Last Updated: Fri May 12, 2023 06:56 PM UTC
Owner: Erich

On Windows reading the first line with linein has a runtime that goes up with the square of the line length, e. g.
10mb 3.7 s
100mb 337.6 s
Our variable line read code extends the read buffer by a small fixed amount each time more space is required which a seemingly inefficient realloc doesn't handle well.

Doubling the read buffer size when more space is needed should fix this (gaining orders of magnitudes for e. g. a one GB linein).

This seems to be a lesser issue on Linux, but the proposed fix will still provide some speed-up.

---

Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/oorexx/bugs/1842/

To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

[Erich]

Committed code fix with revision [r12511].