Running speed is slow if user 'List' instead of open()

[Chris]

I wrote a program which read a file (call a) line by line and use this content to search some files (call b) inside a folder, my first version is 2 loop, first loop is openning b and open a each time to search. The second version is open b and save all content to List, and then 2 loop for openning b and the list, I suppose the list will be faster because no file open/read opeartion each time.

However, the speed is slower than the first version very much. Is file open / read is faster than access the list or my program is not good in list using?

Thanks
Chris

Version 1, open a each time to search b

#coding: UTF-8
import codecs
import os
import sys
notepad.runPluginCommand('Python Script', 'Show Console')

filePathSrc = u'C:\\Temp\\a'

# Subsitution File
subsitutionFile = u'C:\\TEMP\\subsitutions.txt'

# Program Start
console.write('Program Start !!\n')
console.write(u'Source: ' + filePathSrc + '\n')
for root, dirs, files in os.walk(filePathSrc):
    console.write('Searching ' + root + '\n')
    for fn in files:
        if fn[-4:] == u'.srt' or fn[-4:] == u'.ssa' or fn[-4:] == u'.ass':
            fileName = root + '\\' + fn
            console.write(u'fileName: ' + fileName + '\n')
            notepad.open(fileName.encode('utf-8'))
# replace value in subsitution file, separate values with space
# Foramat
# A B
            with open(subsitutionFile) as f:
                replaceCount = 0
                for l in f:
                    #console.write (l + '\n')
                    s = l.split()
                    if len(s) > 1:
                        startPos = 0
                        #Display the result first, replace if found
                        while True:
                            pos = editor.findText(FINDOPTION.REGEXP, startPos, editor.getLength(), s[0])
                            if pos is None:
                                break
                            else:
                                if startPos == 0:
                                    console.write('from:' + s[0] + '\t to:' + s[1] + '\n')
                                editor.gotoPos(pos[0])
                                console.write(str(editor.lineFromPosition(editor.getCurrentPos()))  + ': ' + editor.getCurLine())
                                startPos = pos[0] + 1
                        if startPos != 0:
                            editor.replace(s[0], s[1])
                            replaceCount += 1
                    else:
                        if len(s) == 1:
                            console.writeError('Only 1 element:' + l)
            f.close()
            if replaceCount > 0:
                notepad.save()
            notepad.close()

Version 2, save the a into list and use for b

#coding: UTF-8
import codecs
import os
import sys
notepad.runPluginCommand('Python Script', 'Show Console')

#Function definition
def SubList(subsitutionFile):
    with open(subsitutionFile) as f:
        slist = []
        for l in f:
            #console.write (l + '\n')
            s = l.split()
            if len(s) > 1:
                sList.append(s)
            else:
                if len(s) == 1:
                    console.writeError('Only 1 element: %s' % l)
    f.close()
    return sList
#End SubList

filePathSrc = u'C:\\Temp\\a'

# Subsitution File
subsitutionFile = u'C:\\TEMP\\subsitutions.txt'

# Program Start
console.write('Program Start !!\n')
console.write('Load Subtitle Replacement List from %s' % subsitutionFile)
sl = SubList(subsitutionFile)
console.write(' Completed\n')
console.write(u'Source: ' + filePathSrc + '\n')
for root, dirs, files in os.walk(filePathSrc):
    console.write('Searching ' + root + '\n')
    for fn in files:
        if fn[-4:] == u'.srt' or fn[-4:] == u'.ssa' or fn[-4:] == u'.ass':
            fileName = root + '\\' + fn
            notepad.open(fileName.encode('utf-8'))
# replace value in subsitution file, separate values with space
# Foramat
# A B
            replaceCount = 0
            for s in sl:
                startPos = 0
                # Find and Display result
                while True:
                    pos = editor.findText(FINDOPTION.REGEXP, startPos, editor.getLength(), s[0])
                    if pos is None:
                        break
                    else:
                        if startPos == 0:
                            console.write('from:' + s[0] + '\t to:' + s[1] + '\n')
                        editor.gotoPos(pos[0])
                        console.write(str(editor.lineFromPosition(editor.getCurrentPos()))  + ': ' + editor.getCurLine())
                        startPos = pos[0] + 1
                # Replace result if found
                if startPos != 0:
                    editor.replace(s[0], s[1])
                    replaceCount += 1
            if replaceCount > 0:
                notepad.save()
            notepad.close()

[CFrank]

To be honest, I don't think that reading a file once and using its content in a list is slower
than opening the substitution file always, if you tested this with more than one file.

But a general question, all that you want to do is to find some text and replace it, correct?

If so, why not using

...
notepad.open(fileName.encode('utf-8'))
for s in sl
    editor.replace(s[0],s[1])
    notepad.save()
    notepad.close()

There is no need for doing findText, saving position ... first.

Another hint, you can define the list outside of the function and
append inside of the function, so there is no need to return the list.
Lists aren't affected by the global/local variable issue.

Cheers
Claudia