Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: [Number Series Copilot] Bug fixes #2106

Open
1 task done
DmitryKatson opened this issue Sep 26, 2024 · 1 comment · Fixed by #2111 or #2251
Open
1 task done

[Bug]: [Number Series Copilot] Bug fixes #2106

DmitryKatson opened this issue Sep 26, 2024 · 1 comment · Fixed by #2111 or #2251
Assignees
Labels
Approved The issue is approved Bug Something isn't working Integration GitHub request for Integration area

Comments

@DmitryKatson
Copy link
Contributor

Describe the issue

Bugs

  1. Bug 1: Incorrect Intent Mapping for "Generate" Command

    • Problem: When users intended to create a new Number Series using the word “generate,” the system often mistakenly interpreted the command as a request to prepare Number Series for the next period (e.g., the next fiscal year). This is a misinterpretation of user intent.
    • Expected Behavior: The LLM should correctly distinguish between creating a new Number Series (using terms like "create" or "generate") and preparing a Number Series for the next year/period (using more specific time-related phrases).
    • Proposed Fix: Update the tools definition and prompt to clarify the distinction between creating new Number Series and generating Number Series for the next period. Ensure that the system identifies the intent based on contextual clues rather than assuming a time-based action when the word "generate" is used.
  2. Bug 2: Incorrect Field Check for Existing Number Series

    • Problem: When a user generated Number Series and some already existed in the database, the LLM incorrectly performed checks on all fields (e.g., warning number, increment number, ending number), which should not be the case for pre-existing series. Only new Number Series should undergo a full check.
    • Expected Behavior: The system should skip validation of non-mandatory fields (such as warning number and increment number) for existing Number Series and only perform these checks on newly created series.
    • Proposed Fix: Modify the behavior to exclude pre-existing Number Series from the field validation checks. Ensure that only mandatory fields (such as Number Series code, description, and series) are validated for new series.
  3. Bug 3: Blank Fields Causing Validation Errors

    • Problem: Sometimes the LLM generated Number Series without values for optional fields like the warning number or increment number. If the user's environment had blank fields in an existing Number Series, the system would replicate this and then fail validation when mandatory fields were checked.
    • Expected Behavior: The system should only validate mandatory fields, such as the Number Series code and description, and allow optional fields to remain blank if necessary. This would prevent errors from being thrown unnecessarily.
    • Proposed Fix: Refine the validation logic so that only mandatory fields are checked. Optional fields, such as warning number and increment number, should not cause validation failures if left blank.

Expected behavior

Steps to Implement:

  1. Bug Fixes:
    • Refactor the LLM prompt and tools definition to improve the understanding of user intent between creating and preparing Number Series for different periods.
    • Update the logic to exclude field checks for existing Number Series and restrict checks to mandatory fields.

Steps to reproduce

Acceptance Criteria:

  • Users can create Number Series without mistakenly triggering next-period Number Series generation when using the word "generate."
  • Existing Number Series are excluded from unnecessary checks, and only mandatory fields are validated for new series.

Additional context

No response

I will provide a fix for a bug

  • I will provide a fix for a bug
@DmitryKatson DmitryKatson added the Bug Something isn't working label Sep 26, 2024
@JesperSchulz JesperSchulz added the Approved The issue is approved label Sep 27, 2024
DmitryKatson added a commit to DmitryKatson/BCApps that referenced this issue Sep 27, 2024
…ation

Summary: This commit resolves three bugs in the Number Series generation process. It improves intent mapping for the "Generate" command, corrects field checks for existing Number Series, and refines validation logic to allow optional fields to be blank. These fixes enhance the accuracy and reliability of Number Series generation.

Fixes microsoft#2106
@JesperSchulz JesperSchulz added the Integration GitHub request for Integration area label Sep 30, 2024
JesperSchulz pushed a commit that referenced this issue Oct 23, 2024
…alidation (#2111)

<!-- Thank you for submitting a Pull Request. If you're new to
contributing to BCApps please read our pull request guideline below
* https://github.com/microsoft/BCApps/Contributing.md
-->

#### Summary <!-- Provide a general summary of your changes -->

This PR addresses three critical bugs in the Number Series Copilot:

1. **Bug 1: Incorrect Intent Mapping for "Generate" Command**
- The system was misinterpreting the "generate" command, often preparing
Number Series for the next period instead of creating a new one. The fix
involves updating the tools definition and prompt to clarify the
distinction between creating new Number Series and generating Number
Series for the next period.

2. **Bug 2: Incorrect Field Check for Existing Number Series**
- The system was incorrectly performing checks on all fields for
existing Number Series. The fix modifies the behavior to exclude
pre-existing Number Series from the field validation checks, ensuring
that only mandatory fields are validated for AI generated series.

3. **Bug 3: Blank Fields Causing Validation Errors**
- The system was generating Number Series without values for optional
fields, causing validation errors. The fix refines the validation logic
so that only mandatory fields are checked, allowing optional fields to
remain blank if necessary.

**Steps to Implement:**

1. Refactor the LLM prompt and tools definition to improve the
understanding of user intent between creating and preparing Number
Series for different periods.
2. Update the logic to exclude field checks for existing Number Series
and restrict checks to mandatory fields.

**Acceptance Criteria:**

- Users can create Number Series without mistakenly triggering
next-period Number Series generation when using the word "generate."
- Existing Number Series are excluded from unnecessary checks, and only
mandatory fields are validated for new series.

These fixes aim to enhance the accuracy and reliability of the Number
Series Copilot, ensuring it correctly interprets user commands and
validates the necessary fields.


#### Work Item(s) <!-- Add the issue number here after the #. The issue
needs to be open and approved. Submitting PRs with no linked issues or
unapproved issues is highly discouraged. -->

Fixes #2106 

Fixes
[AB#550892](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/550892),
[AB#544073](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/544073)
JesperSchulz pushed a commit that referenced this issue Oct 23, 2024
…alidation (#2111)

<!-- Thank you for submitting a Pull Request. If you're new to
contributing to BCApps please read our pull request guideline below
* https://github.com/microsoft/BCApps/Contributing.md
-->

#### Summary <!-- Provide a general summary of your changes -->

This PR addresses three critical bugs in the Number Series Copilot:

1. **Bug 1: Incorrect Intent Mapping for "Generate" Command**
- The system was misinterpreting the "generate" command, often preparing
Number Series for the next period instead of creating a new one. The fix
involves updating the tools definition and prompt to clarify the
distinction between creating new Number Series and generating Number
Series for the next period.

2. **Bug 2: Incorrect Field Check for Existing Number Series**
- The system was incorrectly performing checks on all fields for
existing Number Series. The fix modifies the behavior to exclude
pre-existing Number Series from the field validation checks, ensuring
that only mandatory fields are validated for AI generated series.

3. **Bug 3: Blank Fields Causing Validation Errors**
- The system was generating Number Series without values for optional
fields, causing validation errors. The fix refines the validation logic
so that only mandatory fields are checked, allowing optional fields to
remain blank if necessary.

**Steps to Implement:**

1. Refactor the LLM prompt and tools definition to improve the
understanding of user intent between creating and preparing Number
Series for different periods.
2. Update the logic to exclude field checks for existing Number Series
and restrict checks to mandatory fields.

**Acceptance Criteria:**

- Users can create Number Series without mistakenly triggering
next-period Number Series generation when using the word "generate."
- Existing Number Series are excluded from unnecessary checks, and only
mandatory fields are validated for new series.

These fixes aim to enhance the accuracy and reliability of the Number
Series Copilot, ensuring it correctly interprets user commands and
validates the necessary fields.


#### Work Item(s) <!-- Add the issue number here after the #. The issue
needs to be open and approved. Submitting PRs with no linked issues or
unapproved issues is highly discouraged. -->

Fixes #2106 

Fixes
[AB#550892](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/550892),
[AB#544073](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/544073)
@JesperSchulz
Copy link
Contributor

Also backporting to 25.x. Reopening.

@JesperSchulz JesperSchulz reopened this Oct 23, 2024
JesperSchulz added a commit that referenced this issue Oct 23, 2024
…ield V… (#2251)

…alidation (#2111)

<!-- Thank you for submitting a Pull Request. If you're new to
contributing to BCApps please read our pull request guideline below
* https://github.com/microsoft/BCApps/Contributing.md -->

#### Summary <!-- Provide a general summary of your changes -->

This PR addresses three critical bugs in the Number Series Copilot:

1. **Bug 1: Incorrect Intent Mapping for "Generate" Command**
- The system was misinterpreting the "generate" command, often preparing
Number Series for the next period instead of creating a new one. The fix
involves updating the tools definition and prompt to clarify the
distinction between creating new Number Series and generating Number
Series for the next period.

2. **Bug 2: Incorrect Field Check for Existing Number Series**
- The system was incorrectly performing checks on all fields for
existing Number Series. The fix modifies the behavior to exclude
pre-existing Number Series from the field validation checks, ensuring
that only mandatory fields are validated for AI generated series.

3. **Bug 3: Blank Fields Causing Validation Errors**
- The system was generating Number Series without values for optional
fields, causing validation errors. The fix refines the validation logic
so that only mandatory fields are checked, allowing optional fields to
remain blank if necessary.

**Steps to Implement:**

1. Refactor the LLM prompt and tools definition to improve the
understanding of user intent between creating and preparing Number
Series for different periods.
2. Update the logic to exclude field checks for existing Number Series
and restrict checks to mandatory fields.

**Acceptance Criteria:**

- Users can create Number Series without mistakenly triggering
next-period Number Series generation when using the word "generate."
- Existing Number Series are excluded from unnecessary checks, and only
mandatory fields are validated for new series.

These fixes aim to enhance the accuracy and reliability of the Number
Series Copilot, ensuring it correctly interprets user commands and
validates the necessary fields.


#### Work Item(s) <!-- Add the issue number here after the #. The issue
needs to be open and approved. Submitting PRs with no linked issues or
unapproved issues is highly discouraged. -->

Fixes #2106 

Fixes
[AB#555649](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/555649)

Co-authored-by: Dmitry Katson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Approved The issue is approved Bug Something isn't working Integration GitHub request for Integration area
Projects
None yet
2 participants