I thinks this is the expected behaviour

2013/5/19 asdf asdf <psdnym11@outlook.com>

I'm measuring the distance between certain residues in an alignment and I've noticed a discrepancy in the selections returned by the "select within command" and the actual distances measured to the atoms returned by the selection. I was wondering if maybe somebody knew where I was going wrong.

When I run the following commands

> fetch 1MM2
> fetch 2kgi
> align 2kgi, 1MM2
> select asdf, 1MM2////CA w. 5 of 2kgi///22/CA

You do not specify what state (http://pymolwiki.org/index.php/State) you want to look at, so the selection takes in any atoms that are within the given distance. In your example the 7th model in 1MM2 is within 5 from the 7th state of 2kgi, so that gets added to the selection.
> iterate asdf, print(resi + " " + resn + " " + name + " " + str(cmd.distance("tmp", "2kgi///22/CA", "1MM2///" + resi + "/CA")))

If I understand correctly, here you just get the distances between the selected atoms, which then gives the output you get.

I get this output:

24 CYS CA 5.15101623535
25 ASP CA 3.34501266479
26 THR CA 6.03605747223
47 GLU CA 3.80576062202
48 TRP CA 3.85521435738
49 LEU CA 4.28281402588

I'm just a little confused about why atoms that are further than 5 angstroms away from the atom I specified in the select command are returned. Is there some difference in the way "select within" and "cmd.distance", or have a confused some of the commands?

Any suggestions are appreciated

It should work as expected if you split the model into states with split_states (http://pymolwiki.org/index.php/Split_States) and then make the selection on just one of those models.

Best regards,
Folmer Fredslund

PS: It's nice to put in you own name

AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
PyMOL-users mailing list (PyMOL-users@lists.sourceforge.net)
Info Page: https://lists.sourceforge.net/lists/listinfo/pymol-users
Archives: http://www.mail-archive.com/pymol-users@lists.sourceforge.net

Folmer Fredslund