To start clean in Blender you can act several ways.
Start with a saved state
You can save scene initial state by press Ctrl + U (or select File > Save user settings). Further in any moment reload it by press Ctrl + N (or select File > New).
Select File > Load Factory Settings.
Delete all objects
For deleting all scene objects press in order hotkeys: A, X, Enter.
A — to select all, X — to delete selected, Enter - to confirm deleting.