Skip to content

Contribution

Thank you for considering contributing to ParsBench! Your contributions will help improve this framework for benchmarking Large Language Models (LLMs) on Persian language tasks. Here are the guidelines for contributing to the project.

How to Contribute

1. Fix Bugs

If you find a bug in ParsBench, you can contribute by:

  • Reporting the bug in the Issue Tracker.
  • Providing a detailed description of the bug, including steps to reproduce it, the expected behavior, and the actual behavior.
  • Submitting a pull request (PR) with a fix for the bug. Please ensure your fix includes:
  • Clear and concise code changes.
  • Relevant tests to verify the fix.
  • Updates to the documentation, if applicable.

2. Add Features

If you have an idea for a new feature, you can contribute by:

  • Proposing the feature in the Issue Tracker.
  • Providing a detailed description of the feature, including its purpose and potential impact.
  • Submitting a PR that implements the feature. Please ensure your implementation includes:
  • Clear and concise code changes.
  • Relevant tests to verify the feature.
  • Updates to the documentation, including usage examples.

3. Add New Tasks with New Datasets

To expand the capabilities of ParsBench, you can contribute by:

  • Proposing a new task in the Issue Tracker.
  • Providing a detailed description of the task and the dataset, including:
  • The task's objective and relevance to Persian language processing.
  • Details about the dataset, including its source, format, and any preprocessing steps.
  • Submitting a PR that adds the new task and dataset. Please ensure your contribution includes:
  • Clear and concise code changes.
  • Integration of the dataset with existing benchmarking tools.
  • Relevant tests to verify the new task.
  • Updates to the documentation, including instructions on how to use the new task.

4. Run Benchmarks on SoTA Models or Fine-tuned Open-source Models

You can help by running benchmarks on state-of-the-art (SoTA) models or fine-tuned open-source models and sharing the results. To contribute:

  • Select a SoTA model or fine-tuned open-source model relevant to Persian language tasks.
  • Run benchmarks using ParsBench.
  • Share the results by submitting a report in the Issue Tracker. Include:
  • Details about the model and any fine-tuning performed.
  • Saved matches.
  • Benchmarking results, including performance metrics and any observations.
  • Suggestions for improvements, if any.

Contribution Process

  1. Fork the Repository: Fork the ParsBench repository to your GitHub account.
  2. Clone the Repository: Clone the forked repository to your local machine.
git clone https://github.com/shahriarshm/parsbench.git
cd ParsBench
  1. Create a Branch: Create a new branch for your contribution.
git checkout -b feature/your-feature-name
  1. Make Changes: Implement your changes in the new branch.
  2. Commit Changes: Commit your changes with a meaningful commit message.
git add .
git commit -m "Description of your changes"
  1. Push Changes: Push your changes to your forked repository.
git push origin feature/your-feature-name
  1. Create a Pull Request: Open a pull request (PR) to the main repository. Provide a detailed description of your changes and any relevant information.

Code Style and Testing

  • Follow the existing code style and conventions used in the project.
  • Write clear and concise code with appropriate comments.
  • Include relevant tests for your contributions.
  • Ensure all tests pass before submitting your PR.

Getting Help

If you have any questions or need help, feel free to reach out in the Issue Tracker.

Thank you for your contributions!