pyfltk-user

[Pyfltk-user] Timer and subprocess problems.

[Benjamin D. <ben...@py...>]

I am using the binary Linux pyfltk v1.1.4 build from the sourceforge
website together with v2.5.2 of Python and v1.1.9 of fltk to create a
GUI wrapper around a command line process to use hdparm to securely
erase harddrives.

To provide user feedback I am using Fl_repeat_timeout and then
Fl.add_timeout to update a status bar and provide an animated image.
Separate threads are called by the main program to process each drive
using thread.start_new_thread().  The threads then invoke the command
line utilities using commands.getstatusoutput().  The utilities do not
immediately return - especially when falling back to the wipe function
- and this causes the script to hang and the progress bar / animation
to freeze giving the user the impression that it has hung.  I have
tried replacing commands.getstatusoutput() with subprocess.call() but
the problem persists.

This is my first time using Python and fltk.  I've tried searching for
more information on python threads but pretty much drawn a blank.
Consequently, I should be grateful for any advice on how to ensure
that the timer function continues to operate.

Ben

Re: [Pyfltk-user] Timer and subprocess problems.

[Stefan Sonnenberg-C. <ste...@py...>]

Am Mo, 18.05.2009, 12:30, schrieb Benjamin Donnachie:
> I am using the binary Linux pyfltk v1.1.4 build from the sourceforge
> website together with v2.5.2 of Python and v1.1.9 of fltk to create a
> GUI wrapper around a command line process to use hdparm to securely
> erase harddrives.
>
> To provide user feedback I am using Fl_repeat_timeout and then
> Fl.add_timeout to update a status bar and provide an animated image.
> Separate threads are called by the main program to process each drive
> using thread.start_new_thread().  The threads then invoke the command
> line utilities using commands.getstatusoutput().  The utilities do not
> immediately return - especially when falling back to the wipe function
> - and this causes the script to hang and the progress bar / animation
> to freeze giving the user the impression that it has hung.  I have
> tried replacing commands.getstatusoutput() with subprocess.call() but
> the problem persists.
>
> This is my first time using Python and fltk.  I've tried searching for
> more information on python threads but pretty much drawn a blank.
> Consequently, I should be grateful for any advice on how to ensure
> that the timer function continues to operate.
>
> Ben
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables
> unlimited royalty-free distribution of the report engine
> for externally facing server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Pyfltk-user mailing list
> Pyf...@li...
> https://lists.sourceforge.net/lists/listinfo/pyfltk-user
>
>
My guess is, that this has nothing to do with pyFLTK.
(Having used heavy threading besides pyFLTK in the same program)
But please post your code, so that someone has a chance to spot
the critical part of your code.

Re: [Pyfltk-user] Timer and subprocess problems.

[Benjamin D. <ben...@py...>]

2009/5/18 Stefan Sonnenberg-Carstens <ste...@py...>:
> My guess is, that this has nothing to do with pyFLTK.
> (Having used heavy threading besides pyFLTK in the same program)

It's looking much more promising with subprocess.Popen!

> But please post your code, so that someone has a chance to spot
> the critical part of your code.

As it's my first attempt at Python the code is rather scrappy and I'd
rather keep it under wraps for now.

Take care,

Ben

Re: [Pyfltk-user] Timer and subprocess problems.

[Stefan Sonnenberg-C. <ste...@py...>]

Am Mo, 18.05.2009, 13:42, schrieb Benjamin Donnachie:
> 2009/5/18 Stefan Sonnenberg-Carstens
> <ste...@py...>:
>> My guess is, that this has nothing to do with pyFLTK.
>> (Having used heavy threading besides pyFLTK in the same program)
>
> It's looking much more promising with subprocess.Popen!
>
>> But please post your code, so that someone has a chance to spot
>> the critical part of your code.
>
> As it's my first attempt at Python the code is rather scrappy and I'd
> rather keep it under wraps for now.
>
> Take care,
>
> Ben
>
>

Everyone started somewhere :-)

Re: [Pyfltk-user] Timer and subprocess problems.

[Benjamin D. <ben...@py...>]

2009/5/18 Stefan Sonnenberg-Carstens <ste...@py...>:
>> As it's my first attempt at Python the code is rather scrappy and I'd
>> rather keep it under wraps for now.
> Everyone started somewhere :-)

True... :)  Well, popen doesn't seem to have helped much after all...

Please don't be too harsh! :)  Here's the thread function:

# drivedetails is an array in the format [device, serial_number] and
threads is a count of active threads.

def eraseThread(lock, offset):
	global drivedetails
	global threads

	# Basic behaviour - try locking drive, try secure erase.
	# Unlock drive.  If secure erase failed, try wipe
	# If wipe failed, stop
	
	# Enable smart
	# Fill drive
	# Check smart status.

	driveErased = 0

#	print "Thread to erase " + drivedetails[offset][0]

#	Fl.lock()
	window.statusBox.buffer().append(drivedetails[offset][0] + ": trying
secure erase\n")
#	Fl.unlock()

	# May fail for drive already locked
#	commands.getstatusoutput('hdparm --user-master u --security-set-pass
PASSWORD /dev/' + drivedetails[offset][0])
	executing = subprocess.Popen ([ "/sbin/hdparm", "--user-master", "u",
 "--security-set-pass", "PASSWORD", "/dev/" + drivedetails[offset][0]
], -1, None, None, None, None, None, False, False, None, None, False)
#	executing.wait()
	while (executing.poll() == None):
		time.sleep(1)


	# Attempt secure erase

#	status = commands.getstatusoutput('hdparm --user-master u
--security-erase-enhanced PASSWORD /dev/' + drivedetails[offset][0])
	executing = subprocess.Popen ([ "/sbin/hdparm", "--user-master", "u",
"--security-erase-enhanced", "PASSWORD", "/dev/" +
drivedetails[offset][0] ], -1, None, None, None, None, None, False,
False, None, None, False)
	executing.wait()

	if (executing.returncode == 0):
		driveErased = 1

	# Ensure drive left in predictable state

#	commands.getstatusoutput('hdparm --user-master u --security-disable
PASSWORD /dev/' + drivedetails[offset][0])
	executing = subprocess.Popen ([ "/sbin/hdparm", "--user-master", "u",
"--security-disable", "PASSWORD", "/dev/" + drivedetails[offset][0] ],
-1, None, None, None, None, None, False, False, None, None, False)
	executing.wait()

	if (driveErased == 0):
#		Fl.lock()
		window.statusBox.buffer().append(drivedetails[offset][0] + ": trying
overwrite\n")
#		Fl.unlock()
		
#		status = commands.getstatusoutput('nice -n 19 wipe -fkQ 2 /dev/' +
drivedetails[offset][0])
#		status = subprocess.call([ "wipe", "-fkQ 2", "/dev/" +
drivedetails[offset][0] ])
		executing = subprocess.Popen ([ "/usr/bin/wipe", "-fkQ", "2",
"/dev/" + drivedetails[offset][0] ], -1, None, None, None, None, None,
False, False, None, None, False)
#		executing.wait()

	if (executing.returncode == 0):
#		Fl.lock()
		window.statusBox.buffer().append(drivedetails[offset][0] + ":
enabling SMART and filling drive\n")
#		Fl.unlock()
		
#		commands.getstatusoutput('smartctl --smart=on /dev/' +
drivedetails[offset][0])
#		commands.getstatusoutput('dcfldd sizeprobe=of textpattern="STRING
TO FILL DRIVE * " of=/dev/' + drivedetails[offset][0])

#		health = commands.getstatusoutput('smartctl --health /dev/' +
drivedetails[offset][0])

		executing = subprocess.Popen ([ "/usr/sbin/smartctl", "--smart=on",
"/dev/" + drivedetails[offset][0] ], -1, None, None, None, None, None,
False, False, None, None, False)
		executing.wait()

#	executing = subprocess.Popen ([ "/usr/bin/dcfldd", "sizeprobe=of",
"textpattern='STRING TO FILL DRIVE * '" , "of=/dev/" +
drivedetails[offset][0] ], -1, None, None, None, None, None, False,
False, None, None, False)
#	executing.wait()

		executing = subprocess.Popen ([ "/usr/sbin/smartctl", "--health",
"/dev/" + drivedetails[offset][0] ], -1, None, None, None, None, None,
False, False, None, None, False)
		executing.wait()

#	Fl.lock()
		window.statusBox.buffer().append(drivedetails[offset][0] + ": complete\n")


		if (executing.returncode == 0):
			window.messagesBox.buffer().append(drivedetails[offset][0] + ":
erased successfully\n")
		else:
			window.messagesBox.buffer().append(drivedetails[offset][0] + ":
REPLACE DRIVE - FAILED SMART\n")

	else:
		window.messagesBox.buffer().append(drivedetails[offset][0] + ":
CHECK DRIVE - FAILED ERASE\n")

#	Fl.unlock()

	lock.acquire()
	threads -= 1
	lock.release()


# *** The erase button callback:

def erase_cb(widget, data):
	global dance
	global drivedetails
	global threads
	
	if window.driveList.nchecked() > 0:
		window.messagesBox.buffer().text("")
		window.statusBox.buffer().text("")
		window.progressBar.activate()
		window.messagesBox.activate()
		window.statusBox.activate()
		window.driveList.deactivate()
		window.eraseButton.deactivate()
		window.menu.deactivate()

		threadsLck = thread.allocate_lock()

		dance = 1
		
		for loop in range(window.driveList.nitems()):
			if (window.driveList.checked(loop+1) == 1):
				if (thread.start_new_thread (eraseThread, (threadsLck, loop))):
					threadsLck.acquire()
					threads += 1
					threadsLck.release()

		while (threads != 0):
#			time.sleep (1)
			pass

		# All threads done

		dance = 0
		window.driveList.check_none()
		window.progressBar.deactivate()
		window.driveList.activate()
		window.eraseButton.deactivate()
		window.menu.activate()


# *** The timer function:

def showNextImage_cb():
	global image
	global dance
	global progress
	global progressDirection
	
	# Only change image if dance enabled - ie erase in progress
	# Callback constantly called.  Better behaviour would be to disable when
	#   not needed.  However, system likely to be idle when not needed, so...
	
	if dance:
		image=image + 1
		if image > 29:
			image = 0
		window.progress.image(penguin_image[image])
		
		progress = progress + progressDirection
		
		if (progress < 2) or (progress > 499):
		  progressDirection = progressDirection * -1
		
		window.progressBar.value(progress)
		window.progressBar.redraw()
		window.progress.redraw()
		
	Fl.repeat_timeout(0.02, showNextImage_cb)


# *** ... and main function:

# main

window = new_view()
drivelist()
window.show(1, sys.argv)

# Register timer for animated image

Fl.add_timeout(0.02, showNextImage_cb)

Fl.run()


I experimented with, while (executing.poll() == None): print "." in
the thread function and the terminal filled with dots, however the GUI
failed to update.

Take care,

Ben

Re: [Pyfltk-user] Timer and subprocess problems.

[Benjamin D. <ben...@py...>]

2009/5/18 Benjamin Donnachie <ben...@py...>:
> def eraseThread(lock, offset):

Oops - some of the logic is screwed.  I was trying to remove unused
and unnecessary variables before posting...

However, the basic concept is sound.

Ben

Re: [Pyfltk-user] Timer and subprocess problems.

[Robert A. <ro...@gm...>]

On Mon, May 18, 2009 at 3:30 AM, Benjamin Donnachie
<ben...@py...> wrote:
> I am using the binary Linux pyfltk v1.1.4 build from the sourceforge
> website together with v2.5.2 of Python and v1.1.9 of fltk to create a
> GUI wrapper around a command line process to use hdparm to securely
> erase harddrives.
>
> To provide user feedback I am using Fl_repeat_timeout and then
> Fl.add_timeout to update a status bar and provide an animated image.
> Separate threads are called by the main program to process each drive
> using thread.start_new_thread().  The threads then invoke the command
> line utilities using commands.getstatusoutput().  The utilities do not
> immediately return - especially when falling back to the wipe function
> - and this causes the script to hang and the progress bar / animation
> to freeze giving the user the impression that it has hung.  I have
> tried replacing commands.getstatusoutput() with subprocess.call() but
> the problem persists.



two ideas.

1) try calling the commands with & on the end (ie "du -sh / &") to
fork the subprocess so it doesn't block
2) use Fl.check() to update the gui for commands that are repeated in
a loop inside a callback.
http://www.fltk.org/doc-1.1/Fl.html#Fl.check


>
> This is my first time using Python and fltk.  I've tried searching for
> more information on python threads but pretty much drawn a blank.
> Consequently, I should be grateful for any advice on how to ensure
> that the timer function continues to operate.
>
> Ben
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables
> unlimited royalty-free distribution of the report engine
> for externally facing server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Pyfltk-user mailing list
> Pyf...@li...
> https://lists.sourceforge.net/lists/listinfo/pyfltk-user
>



-- 
Robert Arkiletian
Eric Hamber Secondary, Vancouver, Canada