#339 Incorrect quotes when executing custom build commands

[Riot]

Trying to debug strange problems with custom build commands, it seems something odd is going on with quotes - the custom build command is not executed in the same way as if it had simply been typed in the bash prompt.

Steps to reproduce:

• Create a project with a single file in it
• Right-click that file, properties, advanced, use custom build command
• Enter bash -c "for line in 'this is' 'a test'; do echo test; done"
• Enable settings -> compiler -> other settings -> show full command line
• Try to build
• The build log shows is a: 1: is a: Syntax error: Unterminated quoted string

If you execute that custom build command in a bash prompt manually, it should return "test" twice (showing the nested quotes are being parsed properly), and the same behaviour should occur in the build log but doesn't.

The result is that when trying to pass variables like $options in quotes to similar custom build commands using bash -c, the command appears to terminate after the first space separating two consecutive options, rendering the custom build command functionality unusable.

This may be related to the fixed issue https://sourceforge.net/p/codeblocks/tickets/249/

[Riot]

Actually, a simpler example:

The custom build command echo 'one two' will just return the output one.

Doing the same with echo "one two" returns correct output.

The use case I'm trying this with is a build command with content the equivalent of bash -c "echo \"one two\"" which similarly cuts the line after the "one".

[Riot]

I hate to bump, but this issue is still causing me a lot of bother - having to maintain a separate branch with a different codeblocks project to work around this for windows and linux targets - and I think this should be a simple bug to fix, as it's just going to be a matter of interpreting quotes incorrectly somewhere.

I've done some further diagnosis - the program occurs on all operating systems, but it manifests in different ways on Windows and Linux.

Steps to reproduce - enable full commandline logging, and set a custom build command for a file to be:

echo 'one two'

On Windows, this will work correctly, producing in the debug log:

one two

On Linux, however, something goes wrong with the quotes, and the only thing produced in the log is:

one

An attempt to work around this can be made on Linux by setting the build command to be:

echo \'one two\'

which on Linux then produces the correct output. However, this workaround is not adequate because it then breaks Windows (in a strange way). Executing the above build command on Windows produces the output:

\one two'

The above is incorrect output for Windows, as well, so this is broken on both OS's.

All of this leads me to believe that codeblocks is parsing the command in some unintended way and removing or appending single quotes at some stage.

I'd very much appreciate it if this could be looked into ASAP, as this affects all of my projects.

[bluehazzard]

I will try to look into it this weekendr

[bluehazzard]

a little update:
1) I cant reproduce the bug on windows....

echo 'one two'
echo "one two"
echo \'one two\'

all give

Execution of 'echo 'one two'' in 'XXX' failed.

this is probably because echo should run in some sort of cmd? or bash? I would like to have your exact steps...
i can run

bash -c "for line in 'this is' 'a test'; do echo test; done"

on windows and i think the output is fine:

test
test

2) On linux (mint 18 codeblocks 16)

echo "one two"

will print

one two

echo 'one two'

will print

one

so there seems to be a bug...
also the command line

bash -c "for line in 'this is' 'a test'; do echo test; done"

outputs:

Syntax error Unterminated quoted string

i think the code line responsibel for this:

plugins\compilergcc\compilergcc.cpp:1316

   if (!cmd->isRun)
    {
        ExpandBackticks(cmd->command);

        // Run the command in a shell, so stream redirections (<, >, << and >>),
        // piping and other shell features can be evaluated.
        if (!platform::windows)
        {
            wxString shell = Manager::Get()->GetConfigManager(_T("app"))->Read(_T("/console_shell"), DEFAULT_CONSOLE_SHELL);
            cmd->command = shell + _T(" '") + cmd->command + _T("'");
        }
    }

[Riot]

Re. being unable to reproduce on windows: Obviously for a command like echo 'one two' to work on Windows, you would have to have some kind of echo binary somewhere in your path, as is common for msys configurations for instance. If you don't have that, just substitute any command that actually exists.

and yep, looks like you located exactly the undesirable code; also not sure why ExpandBackticks would be desirable here either?

[bluehazzard]

on linux the command executed from codeblocks is:

/bin/sh -c 'bash -c "for line in 'this is' 'a test'; do echo test;done"'

so this seems to be wrong because i get

is a: 1: is a: Syntax error: Unterminated quoted string

but the command is not modified from codelocks, so it seems to be a general problem?
can someone with more experience in shell things tell me how the command should look like?

[EDIT:]
Ok i begin to see the problme:

echo 'one two' 

gets expanded to

/bin/sh -c 'echo 'one two''

this is obviously wrong escaped...

[Riot]

Regarding:

/bin/sh -c 'bash -c "for line in 'this is' 'a test'; do echo test;done"'

Of course this is never going to work; the first single quote inside the command closes the new outside single quotes.

If you're going to be passing a string to sh in single quotes like that, you must first escape all other single quotes in that string.

However, I'm surprised that codeblocks would execute the command in such a clumsy way, explicitly specifying the shell and trying to manually quote the command. If this is what's wanted, then why not simply call the command directly without prepending the shell, or with system(whatever)? Otherwise, popen or equivalent should be used to call the shell and pass the arguments. The method used currently doesn't make sense in any situation.

[bluehazzard]

However, I'm surprised that codeblocks would execute the command in such a clumsy way, explicitly specifying the shell and trying to manually quote the command. If this is what's wanted, then why not simply call the command directly without prepending the shell, or with system(whatever)

it is made so that you can choose what shell you want to use. I don't know what system() calls on unix...
I think the better way to solve this is to escape the quotes correctly...

[bluehazzard]

ok, here is a fix... This should work for unix and the single quote problem

[Teodor Petrov]

This is too simple fix. You need to try harder. Like test if the quote is not already escaped for example.

And I'm not sure that all shell have the same quotation rules. So I'm not sure that it is even possible to fix this in a portable way.

[bluehazzard]

I am not familiar with the whole bash/shell script things, but for what i think even escaped quotes should work... For every single quote you add a escaped single quote. On the end it should be correct escaped. As i said i am not familiar with this so i can't imagine any crazy test case...

[Riot]

I still feel the best way to deal with this would be to give the user better control over where their command is being sent and let them deal themselves with any escaping; all of this trying to guess a shell and trying to guess how it should be called is counterproductive. Simply sending the user's command directly to a system() call would be the simplest, and most portable method which then lets the user deal with their local shell choice themselves.