Although Python is a not a compiled/low-level language, there are certain optimisations that will make it reduce memory usage, become more efficient, and approximate zero-copying found in lower level languages. The best way to do this is to dive right in and see what changes we can make to 'naive' programs. TLDR;
tail tool is a program that, given
some input, extracts only the last N lines from it,
and prints it to the user. A trivial solution would be
something like the following:
import sys def main(): n = int(sys.argv) lines = sys.stdin.readlines() for line in lines[-n:]: print(line, end='') if __name__ == '__main__': main()
The way we'll run our program on Unix systems is:
$ cat head.py | python head.py 10
I think you can find a windows equivalent using pipes and what not. Skipping the deails: the code is quite straightforward, really -
stdinand stores it in a list
Notice that since we only have to display the last N elements, we don't really have to hold all the input in memory at once, we can really just hold the last N elements in memory:
import sys def main(): n = int(sys.argv) q =  for line in sys.stdin:3 q.append(line) if len(q) > n:1 del q2 for line in q: print(line, end='') if __name__ == '__main__': main()
Although a little longer and uglier, the code still does what we set out to do quite elegantly. Let's walk through the important points:
Perhaps the most important change is 2. It reduces our storage requirements from O(size of input) to O(N+1). However this will not be possible without 3. When we use the generator version, what the runtime does for us is this:
while True: try: next_item = get_next_value() except StopIteration: break # do things here
There's an article that explains generators much better than I can, and goes into more detail as well.
Aren't pops/deletes from the 'head' of a list way more expensive than deletes from the end? Yes. Is there another datastructure that can support fast pops and pushes from both ends? Yes.
collections.deque. An often
overlooked datastructure found in Python's standard library.
Also it supports more arguments than meets the average, non
from collections import deque import sys def main(): n = int(sys.argv) for line in deque(sys.stdin, maxlen=n):1 print(line, end='') if __name__ == '__main__': main()
does is that it restricts the size of the deque to at most
N elements. It does it by removing the front/rear
elements to make way for elements added to the rear/front.