fix(MainPipe): fix error report valid when Atomics and SBuffer request miss (#4572)

* Sbuffer write and Atomics should not report errors, but refill from L2
should report ecc error, but requests in

fix(MainPipe): fix error report valid when Atomics and SBuffer request miss (#4572)

* Sbuffer write and Atomics should not report errors, but refill from L2
should report ecc error, but requests in MissQueue carry `isAmo` or
`isStore` and `req.miss` in a request, hence `(s2_req.isAMO ||
s2_req.isStore)` includes the refill, so the missing request of `isAmo`
or `isStore` will not report an error

show more ...

fix(MainPipe): fix `s1_way_en` logic when pseudo tag error inject (#4573)

* `s1_way_en` should use `io.pseudo_error.valid` on the same stage, not
`RegNext(io.pseudo_error.valid)`. Otherwise, `MainPi

fix(MainPipe): fix `s1_way_en` logic when pseudo tag error inject (#4573)

* `s1_way_en` should use `io.pseudo_error.valid` on the same stage, not
`RegNext(io.pseudo_error.valid)`. Otherwise, `MainPipe` may use wrong
way enable to write DCache, it will result in two identical tags.

show more ...

fix(LLPTW): dup_wait_resp should not send last_hptw_req when excp (#4596)

In the original design, the condition for `to_last_hptw_req` was:
`dup_wait_resp && entries(io.mem.resp.bits.id).req_info.s2

fix(LLPTW): dup_wait_resp should not send last_hptw_req when excp (#4596)

In the original design, the condition for `to_last_hptw_req` was:
`dup_wait_resp && entries(io.mem.resp.bits.id).req_info.s2xlate ===
allStage`. As a result, when a newly entered LLPTW request dups with an
entry currently returning from memory, and the new request is marked as
allStage, the `to_last_hptw_req` signal would be true. This causes the
state machine to transition to the `state_last_hptw_req` state and send
a request to HPTW.

However, if the page table returned from memory contains a `vsStagePf`
or `gStagePf`, it should directly go to `mem_out` or `bitmap_check`
without performing a final HPTW translation. Therefore, this commit
fixes the bug by adding a restriction to the original `to_last_hptw_req`
condition to ensure that no exceptions are present; otherwise, the state
machine will transition to either `mem_out` or `bitmap_check`.

Additionally, this PR also fixes a bug where `last_hptw_req_ppn` did not
account for the napot case.

show more ...

fix(LLPTW): dup entry should consider s2xlate in need_to_waiting_vec (#4597)

fix(TLB): explicitly specify the signal width again when truncated (#4588)

Similar to https://github.com/OpenXiangShan/XiangShan/pull/4471

fix(TLB): onlyStage1 req should use s1_paddr rather than s2_paddr (#4587)

In the previous design, `s2_paddr` was used whenever virtualization was
enabled (`s2xlate =/= noS2xlate`). This was incorrec

fix(TLB): onlyStage1 req should use s1_paddr rather than s2_paddr (#4587)

In the previous design, `s2_paddr` was used whenever virtualization was
enabled (`s2xlate =/= noS2xlate`). This was incorrect — we should use
`s2_paddr` only when `onlyStage2` or `allStage` is active, and use
`s1_paddr` when in `onlyStage1` or `noS2xlate` mode. This commit fixes
that bug.

show more ...

fix(PTW): false positive accessFault should not use af_level when resp (#4586)

In certain cases where a `pageFault` or `guestFault` occurs,
`accessFault` signal might still be true; however, it is a

fix(PTW): false positive accessFault should not use af_level when resp (#4586)

In certain cases where a `pageFault` or `guestFault` occurs,
`accessFault` signal might still be true; however, it is actually
invalid and should not be reported. We fixed this bug in commit
https://github.com/OpenXiangShan/XiangShan/pull/4540.

However, in the previous design, the level field of the PTW response was
defined as: `Mux(accessFault, af_level, Mux(guestFault, gpf_level,
level))`. As a result, although we fixed the false accessFault reporting
in https://github.com/OpenXiangShan/XiangShan/pull/4540, the level in
the PTW response was still incorrectly set to `af_level`. This commit
fixes that issue.

Additionally, this commit extracts the arguments in `ptw_resp.apply`
into separate variables to improve code readability. Previously, it was
incorrectly assumed that `pte_valid` was a required condition for
`guestFault`, using the condition: `!(pte_valid && (pageFault ||
guestFault))`. In fact, only `pageFault` needs to consider `pte_valid`;
`guestFault` does not depend on it. This bug is also fixed in this
commit.

show more ...

fix(L2TlbPrefetch): fix flush condition of L2 TLB Prefetch (#4541)

All components within the L2 TLB should use the same flush condition to
avoid potential issues.

fix(PTW): fix exception gen when both af and (pf | gpf) occur (#4540)

When `pte_valid` is true and a page fault or guest page fault occurs,
the original design only treated `ppn_af` as invalid (with

fix(PTW): fix exception gen when both af and (pf | gpf) occur (#4540)

When `pte_valid` is true and a page fault or guest page fault occurs,
the original design only treated `ppn_af` as invalid (without checking
whether the higher bits of ppn are zero). However, in this case, the PMP
check would still be performed, potentially raising the `accessFault`
signal.

This commit fixes the bug by ensuring that if a PMP check fails, only
`accessFault` is raised, and pf or gpf will not be incorrectly asserted.
Therefore, when either pf or gpf is valid, any `accessFault` resulting
from PMP should be ignored.

show more ...

fix(PTWCache): hfence_gvma should ignore g bit (#4539)

In "RISC-V Instruction Set Manual: Volume II: Privileged Architecture":
The G bit in all G-stage PTEs is reserved for future standard use. Unti

fix(PTWCache): hfence_gvma should ignore g bit (#4539)

In "RISC-V Instruction Set Manual: Volume II: Privileged Architecture":
The G bit in all G-stage PTEs is reserved for future standard use. Until
its use is defined by a standard extension, it should be cleared by
software for forward compatibility, and must be ignored by hardware.

Co-authored-by: SpecialWeeks <[email protected]>

show more ...

fix(LLPTW): each LLPTW entry should use its own s2xlate (#4510)

In the previous design, the input s2xlate signal was directly used to
determine whether to virtualize, but the input signal changed to

fix(LLPTW): each LLPTW entry should use its own s2xlate (#4510)

In the previous design, the input s2xlate signal was directly used to
determine whether to virtualize, but the input signal changed to the
default value 0 due to timing problems, resulting in the use of the
wrong PBMTE.

In fact, LLPTW can handle both virtualized and non-virtualized requests
simultaneously. This information is stored in
entries(i).req_info.s2xlate. By using this signal, we can distinguish
between PBMTEs under different virtualization modes. This commit fixes
the bug.

Co-authored-by: SpecialWeeks <[email protected]>

show more ...

Revert "fix(TLB): should always send onlyS1 req when req_need_gpa (#4… (#4551)

…513)"

This reverts commit 6d07e62cded1f9718c229f7c38d297fed2c95cb8.

At the same time, this commit also fixes a bug w

Revert "fix(TLB): should always send onlyS1 req when req_need_gpa (#4… (#4551)

…513)"

This reverts commit 6d07e62cded1f9718c229f7c38d297fed2c95cb8.

At the same time, this commit also fixes a bug where a onlyS1 request
was issued when `req_need_gpa` was active. In fact, even when
`req_need_gpa` is active, it is necessary to determine whether to issue
allStage, onlyS1, or onlyS2 based on the values of the vsatp or hgatp
registers. The previous approach in
https://github.com/OpenXiangShan/XiangShan/pull/4513 was incorrect, and
the details are as follows:

The process of two-stage address translation:

vaddr -> VS-L2 -> G-L2 -> G-L1 -> G-L0 (G-Stage for VS-Stage)
-> VS-L1 -> G-L2 -> G-L1 -> G-L0 (G-Stage for VS-Stage)
-> VS-L0 -> G-L2 -> G-L1 -> G-L0 (G-Stage for VS-Stage)
-> gpaddr -> G-L1 -> G-L0 -> paddr (last G-Stage)

When a page fault occurs at "G-Stage for VS-Stage" in the diagram, the
corresponding VS-Stage result before the arrow is the gpaddr value to be
written to the *tval register.

However, for example, the required gpaddr comes from VS-L0. Although the
first G-Stage query result for VS-L0 is not needed, it is clear that two
G-Stage requests are required before this, after VS-L2 and VS-L1, in
order to obtain the correct VS-L0 memory access address. If the L1 TLB
directly issues an onlyS1 request, then any G-Stage requests will be
ignored, which is unreasonable.

show more ...

fix(MMU): fix gvpn generate when PTWCache Stage1Hit a napot entry (#4527)

For allStage case, when there is a PageTableCache Stage1Hit, the GVPN is
reconstructed within the PTW. However, this reconst

fix(MMU): fix gvpn generate when PTWCache Stage1Hit a napot entry (#4527)

For allStage case, when there is a PageTableCache Stage1Hit, the GVPN is
reconstructed within the PTW. However, this reconstruction process did
not account for the napot case. This commit fixes the bug.

show more ...

fix(LLPTW): should not check g-stage pf when vs-stage pf occured (#4525)

When vs-stage pagefault occurs, check high bits of gvpn for g-stage is
meaningless, and should not report guest page fault.

fix(PTW): should not do pmp check before G-stage finish (#4524)

In https://github.com/OpenXiangShan/XiangShan/pull/4422, For the
virtualized allStage scenario, we attempt to suppress PMP checks unti

fix(PTW): should not do pmp check before G-stage finish (#4524)

In https://github.com/OpenXiangShan/XiangShan/pull/4422, For the
virtualized allStage scenario, we attempt to suppress PMP checks until
the G-stage translation result is returned. However, the approach used
in https://github.com/OpenXiangShan/XiangShan/pull/4422 is to try
ignoring the `accessFault` result when `io.hptw.req.valid` is asserted.

In reality, the `accessFault` signal remains valid because it is
implemented as a `RegEnable`. Therefore, we need to prevent
`sent_to_pmp` from being asserted when sending the G-stage translation
request to HPTW. To address this, we introduce the `vs_finish` signal.
When `vs_finish` is `true.B`, `sent_to_pmp` will be false, thus avoiding
false `accessFault` reports.

show more ...

fix(TLB): should always send onlyS1 req when req_need_gpa (#4513)

`req_need_gpa` indicates that the request needs to access the guest
physical address (GPA) where a GPF occurred by querying the L2 T

fix(TLB): should always send onlyS1 req when req_need_gpa (#4513)

`req_need_gpa` indicates that the request needs to access the guest
physical address (GPA) where a GPF occurred by querying the L2 TLB. When
`req_need_gpa` is valid, it is sufficient to send an onlyStage1 request
to obtain the guest physical address.

In the original design, multiple cases in the `MuxCase` could be valid
at the same time. For example, when both `csr.vsatp.mode =/= 0.U` and
`csr.hgatp.mode =/= 0.U` are valid, an allStage request would be sent
directly, without considering the `req_need_gpa` condition. This commit
fixes that bug.

show more ...

fix(Uncache): uncache mm store needs difftest to update goldenmem (#4470)

feat(Sram): add `SRAM_CTL` interface (#4474)

* add `SRAM_CTL` interface for SRAMTemplate
* use `SRAM_WITH_CTL` to enable,
e.g. `make sim-verilog CONFIG=KunminghuV2Config RELEASE=1
SRAM_WITH_CTL=

feat(Sram): add `SRAM_CTL` interface (#4474)

* add `SRAM_CTL` interface for SRAMTemplate
* use `SRAM_WITH_CTL` to enable,
e.g. `make sim-verilog CONFIG=KunminghuV2Config RELEASE=1
SRAM_WITH_CTL=1`

show more ...

fix(MMU): unify vmid matching logic (#4484)

In scenarios where virtualisation is turned on, either onlyS1, onlyS2,
or allStage, vmid matching is required. Specifically:

onlyS2: s2.vmid
onlyS1 or al

fix(MMU): unify vmid matching logic (#4484)

In scenarios where virtualisation is turned on, either onlyS1, onlyS2,
or allStage, vmid matching is required. Specifically:

onlyS2: s2.vmid
onlyS1 or allStage: s1.vmid

show more ...

fix(PTW): fix gpf_level generate logic when Sv39 (#4482)

For historical reasons, `gpf_level` needs to be 1 greater than
`af_level`. In the original design, only the Sv48 case was considered,
and a s

fix(PTW): fix gpf_level generate logic when Sv39 (#4482)

For historical reasons, `gpf_level` needs to be 1 greater than
`af_level`. In the original design, only the Sv48 case was considered,
and a similar treatment is needed for the Sv39 case.

As an extra note here, `gpf_level` is a 2-bit register, so when
`gpf_level` = 0.U, `gpf_level` - 2.U = `00` - 2 = `10` = 2.U. The PTW
module's `gpf_level` computation uses this feature, which is extremely
unscalable, and, except for the original author, is completely
incomprehensible, and is in dire need of refactoring.

show more ...

fix(PTW): fix gvpn check when req first enter PTW (#4481)

In the previous design, we used `gvpn_gpf` to detect whether the G-Stage
meets the gvpn high level of 0 after the VS-Stage translation is
co

fix(PTW): fix gvpn check when req first enter PTW (#4481)

In the previous design, we used `gvpn_gpf` to detect whether the G-Stage
meets the gvpn high level of 0 after the VS-Stage translation is
completed in allStage case, or else a guest page fault is reported.

However, due to some historical reasons, `gvpn_gpf` can only be used to
detect guest faults during the translation process; for the first time
when entering the PTW after the request to query the Page Table Cache
has finished, similar error checking will be done in
`check_gpa_high_fail`, but when the PTW resp valid is 1, the error
checking will be done in `check_gpa_high_fail`, `full_gvpn` has been
cleared to 0 (line 374), so `gvpn_gpf` will not be 1 (should be 1),
resulting in an error in the returned message.

I do not know why it was designed this way at the time, so I had to add
another `first_gvpn_check_fail` signal to indicate that the first
request into the PTW didn't satisfy the gvpn's high level check, and
therefore needed to report a guest page fault.

show more ...

fix(MainPipe): fix error valid generate logic (#4480)

* Atomics and Sbuffer write request should not report error

fix(MainPipe): fix tag match logic when ecc inject occur (#4479)

* When ecc injection occurs, it is necessary to determine whether there
is a hit cacheline in the `TagArray`. For example, `prefetch.

fix(MainPipe): fix tag match logic when ecc inject occur (#4479)

* When ecc injection occurs, it is necessary to determine whether there
is a hit cacheline in the `TagArray`. For example, `prefetch.w` will be
refilled due to permission mismatch. At this time, there is a hit
cacheline in the `TagArray`. If there is a hit way, it is necessary to
select the way, otherwise select the way determined by replacement.

* It's necessary to determine whether there is a hit with a non-toggle
tag.

show more ...

fix(LLPTW): Should consider napot scenario when allStage (#4473)

In previous design, we always use `ptes(index).getPPN()` to generate PPN
for last G-stage translate. However, when VS-Stage is napot,

fix(LLPTW): Should consider napot scenario when allStage (#4473)

In previous design, we always use `ptes(index).getPPN()` to generate PPN
for last G-stage translate. However, when VS-Stage is napot, we should
use the low 4 bits of vpn for generating ppn.

show more ...

fix(PTW): Should not do gvpn check when pageFault or ppn_af (#4472)

when pageFault or ppn_af valid, it means that an exception occurs when
Stage1 translation. So should not check high bits of gvpn(S

fix(PTW): Should not do gvpn check when pageFault or ppn_af (#4472)

when pageFault or ppn_af valid, it means that an exception occurs when
Stage1 translation. So should not check high bits of gvpn(Stage2 check)

show more ...