## Installation
+Python 3.10+ is required to install `waybacktweets`.
+
```shell
-pip install waybacktweets
+pipx install waybacktweets
```
+[Read more about the installation options](https://waybacktweets.claromes.com/installation).
+
## CLI
```shell
-Usage: waybacktweets [OPTIONS] USERNAME
-
+Usage:
+ waybacktweets [OPTIONS] USERNAME
USERNAME: The Twitter username without @
Options:
- -c, --collapse [urlkey|digest|timestamp:XX]
+ -c, --collapse [urlkey|digest|timestamp:xx]
Collapse results based on a field, or a
substring of a field. XX in the timestamp
value ranges from 1 to 14, comparing the
-v, --verbose Shows the log.
--version Show the version and exit.
-h, --help Show this message and exit.
-
- Examples:
-
- Retrieve all tweets: waybacktweets jack
-
- With options and verbose output: waybacktweets --from 20200305 --to 20231231 --limit 300 --verbose jack
-
- Documentation:
-
- https://waybacktweets.claromes.com/
+Examples:
+ waybacktweets jack
+ waybacktweets --from 20200305 --to 20231231 --limit 300 --verbose jack
+Repository:
+ https://github.com/claromes/waybacktweets
+Documentation:
+ https://waybacktweets.claromes.com
```
## Module
:prog: waybacktweets
:nested: full
+Examples
+---------
+
+``waybacktweets jack``
+
+``waybacktweets --from 20200305 --to 20231231 --limit 300 --verbose jack``
+
+``waybacktweets -f 20200305 -t 20231231 -l 300 -v jack``
+
Collapsing
------------
**It is compatible with Python versions 3.10 and above.**
+Using pipx
+------------
+
+ .. code-block:: shell
+
+ pipx install waybacktweets
+
Using pip
------------
.. code-block:: shell
- pip install waybacktweets
+ pip3 install waybacktweets
Using Poetry
------------
.. code-block:: shell
- pip install poetry
+ pip3 install poetry
**Install the dependencies:**
.. code-block:: shell
- poetry run waybacktweets [SUBCOMMANDS]
+ poetry run waybacktweets [OPTIONS] USERNAME
**Run the Streamlit App:**
.. code-block:: shell
make clean html
-
-`Read the Poetry CLI documentation <https://python-poetry.org/docs/cli/>`_.
readme = "README.md"
repository = "https://github.com/claromes/waybacktweets"
keywords = [
- "twitter",
+ "Twitter",
"X",
"tweet",
- "internet-archive",
- "wayback-machine",
- "osint-tools",
- "osint",
- "command-line",
+ "Internet Archive",
+ "Wayback Machine",
+ "OSINT",
+ "SOCMINT",
]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Topic :: Software Development",
"Topic :: Utilities",
]
-exclude = ["app/**", "assets/**", "docs/**", ".streamlit/**"]
+exclude = ["app/**", "legacy_app/**", ".streamlit/**", "assets/**", "docs/**", "build/**", ".github/**", ".git/**", ".gitignore", ".pre-commit-config.yaml", "CITATION.cff"]
[tool.poetry.urls]
"Homepage" = "https://waybacktweets.claromes.com/"
[tool.flake8]
max-line-length = 88
-extend-ignore = ["E203", "E701"]
+extend-ignore = ["E203", "E701", "E501"]
[tool.poetry.scripts]
waybacktweets = 'waybacktweets._cli:main'
PACKAGE_NAME = "waybacktweets"
+class CustomCommand(click.Command):
+ """
+ Custom Click command that overrides the default help message.
+ """
+
+ def format_help(
+ self, ctx: click.Context, formatter: click.HelpFormatter
+ ) -> None: # noqa: E501
+ """
+ Customize the help message shown when the command is invoked with --help.
+
+ Args:
+ ctx (click.Context): The Click context for the command.
+ formatter (click.HelpFormatter): The formatter used to generate the help text.
+ """ # noqa: E501
+ formatter.write_heading("Usage")
+ formatter.write_text(f" {PACKAGE_NAME} [OPTIONS] USERNAME")
+ formatter.write_text(" USERNAME: The Twitter username without @")
+
+ self.format_options(ctx, formatter)
+
+ formatter.write_heading("Examples")
+ formatter.write_text(" waybacktweets jack")
+ formatter.write_text(
+ " waybacktweets --from 20200305 --to 20231231 --limit 300 --verbose jack"
+ )
+
+ formatter.write_heading("Repository")
+ formatter.write_text(" https://github.com/claromes/waybacktweets")
+
+ formatter.write_heading("Documentation")
+ formatter.write_text(" https://waybacktweets.claromes.com")
+
+
def _parse_date(
ctx: Optional[Any] = None, param: Optional[Any] = None, value: Optional[str] = None
) -> Optional[str]:
Returns:
The input date string formatted in the "YYYYMMDD" format, or None if no date string was provided.
- """ # noqa: E501
+ """
try:
if value is None:
return None
@click.command(
- context_settings={"help_option_names": ["-h", "--help"]},
- epilog="""
-Examples:\n
- Retrieve all tweets: waybacktweets jack\n\n
- With options and verbose output: waybacktweets --from 20200305 --to 20231231 --limit 300 --verbose jack\n\n
-
-Documentation:\n
- https://waybacktweets.claromes.com/
- """, # noqa: E501
+ cls=CustomCommand, context_settings={"help_option_names": ["-h", "--help"]}, help=""
)
@click.argument("username", type=str)
@click.option(
verbose: Optional[bool],
) -> None:
"""
- USERNAME: The Twitter username without @
- """
+ Handles CLI queries for archived tweets with filtering and pagination.
+
+ Args:
+ username (str): Twitter username to search (without the @ symbol).
+ collapse (Optional[str]): Collapse results based on a specific field or a substring
+ of a field. Possible values include 'urlkey', 'digest', or 'timestamp:XX', where
+ XX is the number of digits to match in timestamps (recommended: 4 or more).
+ timestamp_from (Optional[str]): Start date for filtering results (format: YYYYMMDD).
+ timestamp_to (Optional[str]): End date for filtering results (format: YYYYMMDD).
+ limit (Optional[int]): Maximum number of results to return.
+ resumption_key (Optional[str]): Resume a previous query using this key (for pagination).
+ matchtype (Optional[str]): Filter by URL match type: 'exact', 'prefix', 'host', or 'domain'.
+ verbose (Optional[bool]): Print verbose logs during execution.
+ """ # noqa: E501
try:
config.verbose = verbose
projects / waybacktweets.git / commitdiff