top-down: align top-down with Gem5 (#2085)

* topdown: add defines of topdown counters enum

* redirect: add redirect type for perf

* top-down: add stallReason IOs

frontend -> ctrlBlock -> de

top-down: align top-down with Gem5 (#2085)

* topdown: add defines of topdown counters enum

* redirect: add redirect type for perf

* top-down: add stallReason IOs

frontend -> ctrlBlock -> decode -> rename -> dispatch

* top-down: add dummy connections

* top-down: update TopdownCounters

* top-down: imp backend analysis and counter dump

* top-down: add HartId in `addSource`

* top-down: broadcast lqIdx of ROB head

* top-down: frontend signal done

* top-down: add memblock topdown interface

* Bump HuanCun: add TopDownMonitor

* top-down: receive and handle reasons in dispatch

* top-down: remove previous top-down code

* TopDown: add MemReqSource enum

* TopDown: extend mshr_latency range

* TopDown: add basic Req Source

TODO: distinguish prefetch

* dcache: distinguish L1DataPrefetch and CPUData

* top-down: comment out debugging perf counters in ibuffer

* TopDown: add path to pass MemReqSource to HuanCun

* TopDown: use simpler logic to count reqSource and update Probe count

* frontend: update topdown counters

* Update HuanCun Topdown for MemReqSource

* top-down: fix load stalls

* top-down: Change the priority of different stall reasons

* top-down: breakdown OtherCoreStall

* sbuffer: fix eviction

* when valid count reaches StoreBufferSize, do eviction

* sbuffer: fix replaceIdx

* If the way selected by the replacement algorithm cannot be written into dcache, its result is not used.

* dcache, ldu: fix vaddr in missqueue

This commit prevents the high bits of the virtual address from being truncated

* fix-ldst_pri-230506

* mainpipe: fix loadsAreComing

* top-down: disable dedup

* top-down: remove old top-down config

* top-down: split lq addr from ls_debug

* top-down: purge previous top-down code

* top-down: add debug_vaddr in LoadQueueReplay

* add source rob_head_other_repay

* remove load_l1_cache_stall_with/wihtou_bank_conflict

* dcache: split CPUData & refill latency

* split CPUData to CPUStoreData & CPULoadData & CPUAtomicData
* monitor refill latency for all type of req

* dcache: fix perfcounter in mq

* io.req.bits.cancel should be applied when counting req.fire

* TopDown: add TopDown for CPL2 in XiangShan

* top-down: add hartid params to L2Cache

* top-down: fix dispatch queue bound

* top-down: no DqStall when robFull

* topdown: buspmu support latency statistic (#2106)

* perf: add buspmu between L2 and L3, support name argument

* bump difftest

* perf: busmonitor supports latency stat

* config: fix cpl2 compatible problem

* bump utility

* bump coupledL2

* bump huancun

* misc: adapt to utility key&field

* config: fix key&field source, remove deprecated argument

* buspmu: remove debug print

* bump coupledl2&huancun

* top-down: fix sq full condition

* top-down: classify "lq full" load bound

* top-down: bump submodules

* bump coupledL2: fix reqSource in data path

* bump coupledL2

---------

Co-authored-by: tastynoob <[email protected]>
Co-authored-by: Guokai Chen <[email protected]>
Co-authored-by: lixin <[email protected]>
Co-authored-by: XiChen <[email protected]>
Co-authored-by: Zhou Yaoyang <[email protected]>
Co-authored-by: Lyn <[email protected]>
Co-authored-by: wakafa <[email protected]>

show more ...

ICache : finish fencei support

ICache : fix compile error & make itlb and pmp port num more configurable

[WIP] ICache: implement new bankedMetaArray & make prefetchPipe num Configurable

Separate Utility submodule from XiangShan (#1861)

* misc: add utility submodule

* misc: adjust to new utility framework

* bump utility: revert resetgen

* bump huancun

IFU: mmio wait until last instruction retiring

* add 1 stage for mmio_state before sending request to MMIO bus
* check whether the last fetch packet commit all its intructions (the
result of executi

IFU: mmio wait until last instruction retiring

* add 1 stage for mmio_state before sending request to MMIO bus
* check whether the last fetch packet commit all its intructions (the
result of execution path has been decided)
* avoid speculative execution to MMIO bus

show more ...

IFU: add ICache ready

IFU/IPrefetch/ReplacePipe: adjust meta/data access

* IFU: ignore ICache access bundle

* ICacheMainPipe: expand meta/data access output to 4 identical vector
output, each output is connected to a co

IFU/IPrefetch/ReplacePipe: adjust meta/data access

* IFU: ignore ICache access bundle

* ICacheMainPipe: expand meta/data access output to 4 identical vector
output, each output is connected to a copied register trigger by FTQ
requests

* IPrefetch/ReplacePipe: expand meta/data access outpu to 4 indentical
vector output, and each output is triggered by the same signal group

show more ...

[WIP]FTQ: add icache req port

* separate ifu req and icache req for timing optimization

* both ifu ftq_req_ready and icache ftq_req_ready depend on each other

* ifu and icache has pc_mem register

[WIP]FTQ: add icache req port

* separate ifu req and icache req for timing optimization

* both ifu ftq_req_ready and icache ftq_req_ready depend on each other

* ifu and icache has pc_mem register

[WIP]ICacheMainPipe: add copied registers

[WIP]ftq: read ftq_pc_mem one cycle ahead, reqs to be copied

[WIP] FTQ: delete outside bypass

show more ...

Sync timing modification of #1681 and #1793 (#1793)

* ldu: optimize dcache hitvec wiring

In previous design, hitvec is generated in load s1, then send to dcache
and lsu (rs) side separately. As

Sync timing modification of #1681 and #1793 (#1793)

* ldu: optimize dcache hitvec wiring

In previous design, hitvec is generated in load s1, then send to dcache
and lsu (rs) side separately. As dcache and lsu (rs side) is far in real
chip, it caused severe wiring problem.

Now we generate 2 hitvec in parallel:

* hitvec 1 is generated near dcache.
To generate that signal, paddr from dtlb is sent to dcache in load_s1
to geerate hitvec. The hitvec is then sent to dcache to generate
data array read_way_en.

* hitvec 2 is generated near lsu and rs in load_s2, tag read result
from dcache, as well as coh_state, is sent to lsu in load_s1,
then it is used to calcuate hitvec in load_s2. hitvec 2 is used
to generate hit/miss signal used by lsu.

It should fix the wiring problem caused by hitvec

* ldu: opt loadViolationQuery.resp.ready timing

An extra release addr register is added near lsu to speed up the
generation of loadViolationQuery.resp.ready

* l1tlb: replace NormalPage data module and add duplicate resp result

data module:
add BankedSyncDataMoudleWithDup data module:
divided the data array into banks and read as Async, bypass write data.
RegNext the data result * #banks. choose from the chosen data.

duplicate:
duplicate the chosen data and return to outside(tlb).
tlb return (ppn+perm) * #DUP to outside (for load unit only)

TODO: load unit use different tlb resp result to different module.
one for lsq, one for dcache.

* l1tlb: Fix wrong vidx_bypass logic after using duplicate data module

We use BankedSyncDataMoudleWithDup instead of SyncDataModuleTemplate,
whose write ports are not Vec.

Co-authored-by: William Wang <[email protected]>
Co-authored-by: ZhangZifei <[email protected]>
Co-authored-by: good-circle <[email protected]>

show more ...

l1tlb: tlb's req port can be configured to be block or non-blocked (#1656)

each tlb's port can be configured to be block or non-blocked.
For blocked port, there will be a req miss slot stored in tl

l1tlb: tlb's req port can be configured to be block or non-blocked (#1656)

each tlb's port can be configured to be block or non-blocked.
For blocked port, there will be a req miss slot stored in tlb, but belong to
core pipeline, which means only core pipeline flush will invalid them.

For another, itlb also use PTW Filter but with only 4 entries.
Last, keep svinval extension as usual, still work.


* tlb: add blocked-tlb support, miss frontend changes

* tlb: remove tlb's sameCycle support, result will return at next cycle

* tlb: remove param ShouldBlock, move block method into TLB module

* tlb: fix handle_block's miss_req logic

* mmu.filter: change filter's req.ready to canEnqueue

when filter can't let all the req enqueue, set the req.ready to false.
canEnqueue after filtering has long latency, so we use **_fake
without filtering, but the filter will still receive the reqs if
it can(after filtering).

* mmu.tlb: change name from BTlbPtwIO to VectorTlbPtwIO

* mmu: replace itlb's repeater to filter&repeaternb

* mmu.tlb: add TlbStorageWrapper to make TLB cleaner

more: BlockTlbRequestorIO is same with TlbRequestorIO, rm it

* mmu.tlb: rm unused param in function r_req_apply, fix syntax bug

* [WIP]icache: itlb usage from non-blocked to blocked

* mmu.tlb: change parameter NBWidth to Seq of boolean

* icache.mainpipe: fix itlb's resp.ready, not always true

* mmu.tlb: add kill sigal to blocked req that needs sync but fail

in frontend, icache,itlb,next pipe may not able to sync.
blocked tlb will store miss req ang blocks req, which makes itlb
couldn't work. So add kill logic to let itlb not to store reqs.

One more thing: fix icache's blocked tlb handling logic

* icache.mainpipe: fix tlb's ready_recv logic

icache mainpipe has two ports, but these two ports may not valid
all the same time. So add new signals tlb_need_recv to record whether
stage s1 should wait for the tlb.

* tlb: when flush, just set resp.valid and pf, pf for don't use it

* tlb: flush should concern satp.changed(for blocked io now)

* mmu.tlb: add new flush that doesn't flush reqs

Sfence.vma will flush inflight reqs and flushPipe
But some other sfence(svinval...) will not. So add new flush to
distinguish these two kinds of sfence signal

morw: forget to assign resp result when ptw back, fix it

* mmu.tlb: beautify miss_req_v and miss_v relative logic

* mmu.tlb: fix bug, when ptw back and bypass, concern level to genPPN

bug: when ptw back and bypass, forgot to concern level(1GB/2MB/4KB)
when genPPN.

by the way: some funtions need ": Unit = ", add it.

* mmu.filter: fix bug of canEnqueue, mixed with tlb_req and tlb.req

* icache.mainpipe: fix bug of tlbExcp's usage, & with tlb_need_back

Icache's mainpipe has two ports, but may only port 0 is valid.
When a port is invalid, the tlbexcp should be false.(Actually, should
be ignored).
So & tlb_need_back to fix this bug.

* sfence: instr in svinval ext will also flush pipe

A difficult problem to handle:
Sfence and Svinval will flush MMU, but only Sfence(some svinval)
will flush pipe. For itlb that some requestors are blocked and
icache doesn't recv flush for simplicity, itlb's blocked ptw req
should not be flushed.
It's a huge problem for MMU to handle for good or bad solutions. But
svinval is seldom used, so disable it's effiency.

* mmu: add parameter to control mmu's sfence delay latency

Difficult problem:
itlb's blocked req should not be abandoned, but sfence will flush
all infight reqs. when itlb and itlb repeater's delay is not same(itlb
is flushed, two cycles later, itlb repeater is flushed, then itlb's
ptw req after flushing will be also flushed sliently.
So add one parameter to control the flush delay to be the same.

* mmu.tlb: fix bug of csr.priv's delay & sfence valid when req fire

1. csr.priv's delay
csr.priv should not be delayed, csr.satp should be delayed.
for excep/intr will change csr.priv, which will be changed at one
instruction's (commit?). but csrrw satp will not, so satp has more
cycles to delay.
2. sfence
when sfence valid but blocked req fire, resp should still fire.
3. satp in TlbCsrBundle
let high bits of satp.ppn to be 0.U

* tlb&icache.mainpipe: rm commented codes

* mmu: move method genPPN to entry bundle

* l1tlb: divide l1tlb flush into flush_mmu and flush_pipe

Problem:
For l1tlb, there are blocked and non-blocked req ports.
For blocked ports, there are req slots to store missed reqs.
Some mmu flush like Sfence should not flush miss slots for outside
may still need get tlb resp, no matter wrong and correct resp.
For example. sfence will flush mmu and flush pipe, but won't flush
reqs inside icache, which waiting for tlb resp.
For example, svinval instr will flush mmu, but not flush pipe. so
tlb should return correct resp, althrough the ptw req is flushed
when tlb miss.

Solution:
divide l1tlb flush into flush_mmu and flush_pipe.
The req slot is considered to be a part of core pipeline and should
only be flushed by flush_pipe.
flush_mmu will flush mmu entries and inflight ptw reqs.
When miss but sfence flushed its ptw req, re-send.

* l1tlb: code clean, correct comments and rm unused codes

* l2tlb: divide filterSize into ifiterSize and dfilterSize

* l2tlb: prefetch req won't enter miss queue. Rename MSHR to missqueue

* l1tlb: when disable vm, ptw back should not bypass tlb and should let miss req go ahead

show more ...

l2tlb: add counter to ptw-filter to avoid l2tlb deadlock & sync sfence to mmu (#1599)

fix some bugs.

1. fix l2tlb dead-lock bug
l2tlb won't merge requests at same addr. It will be blocked when h

l2tlb: add counter to ptw-filter to avoid l2tlb deadlock & sync sfence to mmu (#1599)

fix some bugs.

1. fix l2tlb dead-lock bug
l2tlb won't merge requests at same addr. It will be blocked when having too many requests.
PtwFilter has a bug that will send too many requests. Add a counter to avoid that.

2. fix sfence sync at mmu
different modules in mmu may get sfence at different latency, which will lost requests or some requests have no receiver.
Sync the sfence latency manually to avoid the bug.

* mmu.filter: add counter not to send to many req to l2tlb

* mmu.filter: fix bug that forget counter signal when block issue and deq

* mmu: set sfence/csr delay to 2 cycle, must sync in mmu

show more ...

ICache: add difftest-Refill test (#1548)

pass reset vector from SimTop (#1545)

bpu: bring bpu control signals into use (#1477)

ITLB <timing>: delay miss and flush req for ITLB (#1457)

* ITLB <timing>: delay miss and flush req for ITLB

* add 2 ILTB requestor and delete tlb_arb

* Bump huancun

* ICacheMainPipe <bug-fi

ITLB <timing>: delay miss and flush req for ITLB (#1457)

* ITLB <timing>: delay miss and flush req for ITLB

* add 2 ILTB requestor and delete tlb_arb

* Bump huancun

* ICacheMainPipe <bug-fix>: fix slot invalid condition

* ITLB <timing>: add port to 6

* ICacheMainPipe <bug-fix>: stop pipe when tlb miss

* ICacheMainPipe <bug-fix>: fix illegal flush

Co-authored-by: LinJiawei <[email protected]>

show more ...

Fetch: optimization timing for IFU/ICache/IPrefetch (#1432)

* IFU <timing>: f2_data select signal optimization

* ICacheMainPipe <timing>: latch fetch req when tlb miss

* Frontend <timing>: add

Fetch: optimization timing for IFU/ICache/IPrefetch (#1432)

* IFU <timing>: f2_data select signal optimization

* ICacheMainPipe <timing>: latch fetch req when tlb miss

* Frontend <timing>: add additional PMP checker

* Ftq <timing>: delete flush condition for prefetch.req

* ICacheMainPipe <timing>: move hit state change to s2

* ICache <bug-fix> delete PMP check assertion

* ICache <bug-fix> fix parity error condition

* ICacheMainPipe <bug-fix>: fix tlb resp condition

* when TLB req has been latched into tlb_slot, the
tlb_all_resp condition, which affects s0_fire should
depend on the slot result.

show more ...

ICache: add parity check enable and prefetch enable control registers (#1406)

* Add Prefetch and Parity enable register for ICache

* Add ICache parity enable control for pipe

IFU: fix fetching RVC instructions from mmio space bug (#1407)

* IFU: fix mmio RVC bug

* IFU: add resend address check for mmio

When a mmio fetch an RVI instruction which cross 64 bits,
IFU

IFU: fix fetching RVC instructions from mmio space bug (#1407)

* IFU: fix mmio RVC bug

* IFU: add resend address check for mmio

When a mmio fetch an RVI instruction which cross 64 bits,
IFU must send paddr + 2.U to fetch the higher 16 bits.
But the paddr + 2.U is not checked by TLB or PMP. This may cause
some unexpected fetch stuck problem.

show more ...

Add simple instruction prefetch for L2 (directed by branch prediction) (#1374)

* Add Naive Instruction Prefetch

* Add instruction prefetch module in ICache

* send Hint to L2 (prefetched data s

Add simple instruction prefetch for L2 (directed by branch prediction) (#1374)

* Add Naive Instruction Prefetch

* Add instruction prefetch module in ICache

* send Hint to L2 (prefetched data stores in L2)

* Ftq: add prefetchPtr and prefetch interface

* Fix IPrefetch PMP Port preempting problem

* Fix merge conflict

show more ...

ICache: raise AF exception when parity wrong or L2 send corrupt (#1376)

* ICache: raise access fault when L2 send corrupt

* ICache: add ECC error connection

* chores: add comments and code cle

ICache: raise AF exception when parity wrong or L2 send corrupt (#1376)

* ICache: raise access fault when L2 send corrupt

* ICache: add ECC error connection

* chores: add comments and code clean-up

* ICache: raise AF when Meta/Data Parity wrong

* Update Frontend.scala

show more ...

csr: optimize exception and trapTarget timing (#1372)

core: delay csrCtrl for two cycles (#1336)

This commit adds DelayN(2) to some CSR-related signals, including
control bits to ITLB, DTLB, PTW, etc.

To avoid accessing the ITLB before control bits

core: delay csrCtrl for two cycles (#1336)

This commit adds DelayN(2) to some CSR-related signals, including
control bits to ITLB, DTLB, PTW, etc.

To avoid accessing the ITLB before control bits change, we also need
to delay the flush for two cycles. We assume branch misprediction or
memory violation does not cause csrCtrl to change.

show more ...

icache: support data/tag r/w op (#1337)

* mem,cacheop: fix read data writeback

* mem,cacheop: rename cacheop state bits

These bits are different from w_*, s_* bits in cache

* mem: enable icache o

icache: support data/tag r/w op (#1337)

* mem,cacheop: fix read data writeback

* mem,cacheop: rename cacheop state bits

These bits are different from w_*, s_* bits in cache

* mem: enable icache op feedback

* icache: update cache op implementation

* chore: remove cache op logic from XSCore.scala

show more ...

core: refactor hardware performance counters (#1335)

This commit optimizes the coding style and timing for hardware
performance counters.

By default, performance counters are RegNext(RegNext(_)).