As part of the Shell Parser challenge on Codecrafters, I had to implement support for single quotes. Anything inside the quotes is taken literally, so spaces a shell would normally collapse are preserved e.g. echo 'hello world' should print 'hello world' with the spaces intact. Two quoted strings sitting next to each other also become a single argument e.g. echo 'hello''world' should print helloworld.
My first stab at it was a for-loop that looped through each character, preserving the spaces inside quotes, and eliminating the rest.
def _parse_quotes(args: str):
output = ''
in_quote = False
previous_char_isspace = False
for char in args:
if char == "'":
in_quote = not in_quote
previous_char_isspace = False
continue
if in_quote:
output += char
continue
if char.isspace():
if previous_char_isspace:
continue
else:
previous_char_isspace = True
else:
previous_char_isspace = False
output += char
return outputEarlier, I used the echo command to illustrate the problem, but the parser needed to work for other commands too. I’d fixated on the echo use-case and ended up solving a smaller problem than the one I actually had.
My solution worked for the echo command, which outputs your arguments as a string. But it didn’t work for the cat command. I realised then that I needed to split the input into arguments, breaking on the quotes and on the spaces that fell outside them.
This was my second for-loop attempt.
def _parse_args(args: str):
output: list[str] = []
arg = ""
in_quote = False
for char in args:
if char == "'":
if not in_quote:
if arg:
output.append(arg)
arg = ""
in_quote = not in_quote
continue
elif in_quote:
arg += char
else:
if arg and char.isspace():
output.append(arg)
output.append(char)
arg = ""
elif arg:
arg += char
if arg:
output.append(arg)
return outputSplitting the input string into items in a list was the right call because the output was easier to wield. However, the code became difficult to read, particularly with the nested conditional logic. Even walking through some of the test cases confused me, which is never a good sign. It is hard to fix what is broken if you don’t know where and how it’s broken.
Although there were still a few failing cases, I could almost taste victory. I realised that there were really only two decisions to make, and therefore two branches:
Is this character a quote? If it is, walk forward until we see the closing quote, and take everything in-between the quotes as part of the current argument.
If it’s not a quote, is it whitespace? If so, the current argument is finished, and we can add it to the list. Otherwise, we add the character to the argument.
Given that we’ll be walking forward, a while loop was the most suitable option.
def _parse_args(args: str):
output: list[str] = []
arg = ""
i = 0
while i < len(args):
if args[i] == "'":
i += 1
while i < len(args) and args[i] != "'":
arg += args[i]
i += 1
i += 1
else:
if args[i].isspace():
output.append(arg)
arg = ""
while i < len(args) and args[i].isspace():
i += 1
else:
arg += args[i]
i += 1
if arg:
output.append(arg)
return outputThe test cases passed, and I was overjoyed.
After completing the challenge, I looked through some of the other solutions on Codecrafters. One of them replaced everything I’d just written with a single call to shlex.
parts = shlex.split(command_line)“Of course,” I thought. “There’s a Python library for this.”
Thank you for reading!