timing(MemBlock): optimize MemBlock timing (#3467)

This PR optimizes the timing of MemBlock. Specific optimizations include
but are not limited to:
+ TLB use the redirect for the next cycle
+ Opt

timing(MemBlock): optimize MemBlock timing (#3467)

This PR optimizes the timing of MemBlock. Specific optimizations include
but are not limited to:
+ TLB use the redirect for the next cycle
+ Optimize VLSU feedback and redirect
+ Optimise ldCancel and writeback signal generation
+ Optimise TLB Query Vaddr/hlv/hlvx/valid etc
+ Delay MMIO Store writeback for 1 Cycle
+ Fix tlbNoQuery and pmp logic
+ Remove clock gating for s3_fast_rep
+ Remove wbq conflict check to LoadPipe/MainPipe
+ Remove Mux in dcache resp data
+ Optimise data generation logic of LoadUnit
+ Duplicate Register in LoadUnit for data writeback
+ Duplicate Register in loadPipe for missQueue enq
+ Add skid buffer in VLSU
+ Select data from metaArray at S1
+ Simplify the enqueuing logic of missQueue
+ Separately generate the ready logic of miss Queue
+ Relax the conditions valid for bankdataArray reads
+ Add Reg between Dcache Mainpipe with sms prefetcher
+ Optimise store exceptionBuffer pipeline

---------

Co-authored-by: weiding liu <[email protected]>
Co-authored-by: Charlie Liu <[email protected]>
Co-authored-by: good-circle <[email protected]>

show more ...

Bump rocket-chip (#2353)

L1 Prefetch (#2261)

* dcache: optimize the ready signal of missqueue

Add a custom arbiter. In the case of multiple sources with the same
cache block address, the arbiter will assign only one ent

L1 Prefetch (#2261)

* dcache: optimize the ready signal of missqueue

Add a custom arbiter. In the case of multiple sources with the same
cache block address, the arbiter will assign only one entry in
misssqueue but ready for all same cache block address requests.

This will reduce the number of replays of the load instruction which cannot
enter the missqueue

* sta, dcache: add A StorePipe in dcache

When the store command passes through the sta pipeline, access the tag
and meta of dcache to determine whether it hits, if it hits, update the
replacement algorithm, and if miss, send a write intent to missqueue

* sta prefetch: add a queue

Enter this queue when the Store Address pipeline sends a request,
determines that it has a cache miss, and the contention for MSHR fails.

The miss request in this queue will be sent to the Store pipeline later.

* sbuffer, dcache: store prefetch burst

A basic implementation of "Boosting Store Buffer Efficiency with
Store-Prefetch Bursts".

Store prefetch at exe is disabled.
Now, when store goes from sq to sbuffer, it will trigger a store
prefetch; when 48 stores fall into 6 cache lines, trigger a store burst
perfetch, which will bring a whole page back into dcache.

* dcache: restric mshr alloc for prefetch req

* restric the max number of entries which can be used by prefetch
* merge two same cache line address prefetch write req
* dynamically detect memset pattern, all mshr can accept prefetch when
pattern is detected

* spb: constantin support

* dcache: fix missqueue prefetch ready

* make prefetch req goes mshr with bigger id

* Revert "spb: constantin support"

This reverts commit 4ee50b89ba4a62cd28fa22d7fbcb2338ad4b1849.

* spb: fix bug in burst generator

* spb: add load prefetch burst support

* topdown: add defines of topdown counters enum

* redirect: add redirect type for perf

* top-down: add stallReason IOs

frontend -> ctrlBlock -> decode -> rename -> dispatch

* top-down: add dummy connections

* top-down: update TopdownCounters

* top-down: imp backend analysis and counter dump

* top-down: add HartId in `addSource`

* top-down: broadcast lqIdx of ROB head

* top-down: frontend signal done

* top-down: add memblock topdown interface

* Bump HuanCun: add TopDownMonitor

* top-down: receive and handle reasons in dispatch

* top-down: remove previous top-down code

* TopDown: add MemReqSource enum

* TopDown: extend mshr_latency range

* TopDown: add basic Req Source

TODO: distinguish prefetch

* store prefetch: refactor parameters and fix bug

* change some parameters
* fix store pipe bug
* fix load prefetch burst

* dcache: distinguish L1DataPrefetch and CPUData

* top-down: comment out debugging perf counters in ibuffer

* TopDown: add path to pass MemReqSource to HuanCun

* TopDown: use simpler logic to count reqSource and update Probe count

* frontend: update topdown counters

* Update HuanCun Topdown for MemReqSource

* top-down: fix load stalls

* top-down: Change the priority of different stall reasons

* store prefetch: add stride and l2 prefetch

* add a stride prefetcher
* spb and stride will issue prefetch to l2
* when store commits, issue a prefetch to l1

* sbuffer: fix eviction

* when valid count reaches StoreBufferSize, do eviction

* spf: change store prefetch structure

* prefetch @ exe -> l2 cache
* stride -> l2 cache

* sbuffer: fix replaceIdx

* If the way selected by the replacement algorithm cannot be written into dcache, its result is not used.

* Revert "sbuffer: fix replaceIdx"

This reverts commit 40c16aca956af9fb32554a0f12d18db41c22eecd.

* spf: find best interval in stamissqueue

* Revert "spf: find best interval in stamissqueue"

This reverts commit d179f0ce15a5ab989a822de7fe48cc5e2cd96914.

* sms: port store to sms

Miss store will train sms like load.

Now, sms will recieve 4 train sources, 2 for miss load, 2 for miss
store, but prefetcher consume 1 train req per cycle, PrefetchTrainFilter
is added to deal with this case.

* bump huancun

* spf: refactor structure

* miss stores will train sms, and send prefetch to l2
* miss stores will send prefetch to l1 on issue or commit
* spb will send prefetch to l1

* memset: fix memset detection

use lqEmpty to check this

* constantin: storepf constantin support

cherry-pick this to use constantin in storepf

* Revert "constantin: storepf constantin support"

This reverts commit 2b97767b9fa757d920cac3d80d4893a1380592c7.

* storepf: add EnableAtCommitMissTrigger

* trigger prefetch at commit only when the store misses with
EnableAtCommitMissTrigger

* bump coupledl2

* prefetch req from L1 to L2 will Acquire T

* fix merge conflict

* storepf: do not read meta&tag when pf is disabled

* storepf: do not read pcMem when sms store is disabled

* fix verilog check

* fix verilog

* missqueue: support merging prefetch

* prefetch req can be merged to pipeline reg
* merging prefetch write will update cmd
* delay sending out acquire when a prefetch write is about to merge

* missqueue: fix bug of merging prefetch write

* delay sending out acquire when a pipeline reg is about to merging a
prefetch write

* temp: disable store pf

* missqueue: disable merging prefetch

* late prefetch will be ignored
* check alias when merging
* enable store pf at issue

* add L1StreamPrefetcher

* fix assert

* let prefetch req prefer loadunit1 more than 0

* stream prefetcher

* disable stream component in SMS, SMS is only trained on real miss
* add a prefetcher monitor to adjust depth & confidence ..
* add L1 L2 stream prefetch

* add gene support

* Revert "add gene support"

This reverts commit 59ae15640ff3d1cc96347f4d3567d48c740a03bb.

* add miss db

* l1pf: add stride & store source info in cache meta

* add a Stride prefetcher and disable Stride component in sms
* prefetch bit in meta is expanded into 3 bits to store source info of
prefetcher

* prefetch: support sending prefetch req to l3

* l1pf: add FDP & refactor

* add basic FDP counters
* change stride from Block addr to Byte addr
* refactor the code

* bump submodules

* disable load related chiseldb to reduce db size

* fix compile

* fix minimalConfig & enable stream

* fix stride pc problem

* fix minimalconfig compile

* bump submodules

* refactor stream stride helper

* fix compile

* bump huancun

* disable db to save size

* fix l2 assert

* bump submodules

---------

Co-authored-by: tastynoob <[email protected]>
Co-authored-by: Haojin Tang <[email protected]>
Co-authored-by: Guokai Chen <[email protected]>
Co-authored-by: XiChen <[email protected]>
Co-authored-by: Zhou Yaoyang <[email protected]>

show more ...

dcache: add optional meta prefetch and access bit

Added meta_prefetch and meta_access related sim perf counter

For now, optional dcache meta prefetch and access can be removed safely

Switch to asynchronous reset for all modules (#1867)

This commit changes the reset of all modules to asynchronous style,
including changes on the initialization values of some registers.
For async

Switch to asynchronous reset for all modules (#1867)

This commit changes the reset of all modules to asynchronous style,
including changes on the initialization values of some registers.
For async registers, they must have constant reset values.

show more ...

dcache: divide meta array into nWays banks (#1723)

It should reduce dcache meta write fanout. Now dcache meta write
actually takes 2 cycles

ErrorArray: optimize timing for read resp (#1670)

dcache: use sram to build ecc array (#1382)

* dcache: use sram to build ecc array

* MainPipe: latch s1_encTag to last until s1_fire

Authored-by: zhanglinjuan <[email protected]>

dcache: let ecc error and l2 corrupt raise load af (#1369)

* dcache: let ecc error and l2 corrupt raise load af

If CSR.smblockctl.cache_error_enable is disabled, ecc error and l2 corrupt
will no

dcache: let ecc error and l2 corrupt raise load af (#1369)

* dcache: let ecc error and l2 corrupt raise load af

If CSR.smblockctl.cache_error_enable is disabled, ecc error and l2 corrupt
will not raise any exception.

* mem: enable cache error by default

* mem: support store ecc check, add ecc error csr

Support store / atom ecc check (early version)
Add ecc error csr to distingush ecc error and other access fault

Timing opt and unit tests to be added.

show more ...

dcache: remove redundant ecc array (#1358)

* dcache: fix bug in ecc check

* dcache: remove redundant ecc array

* CacheInstruction: fix typo

* dcache: fix bugs in cache instruction on ecc

dcache: remove redundant ecc array (#1358)

* dcache: fix bug in ecc check

* dcache: remove redundant ecc array

* CacheInstruction: fix typo

* dcache: fix bugs in cache instruction on ecc

* MetaArray: wrap ecc array as a single module

show more ...

dcache: fix bug in ecc check (#1349)

mem: add CSR based l1 cache instructions (#1116)

New DCache (#1111)

* L1D: provide independent meta array for load pipe

* misc: reorg files in cache dir

* chore: reorg l1d related files

* bump difftest: use clang to compile verialted file

New DCache (#1111)

* L1D: provide independent meta array for load pipe

* misc: reorg files in cache dir

* chore: reorg l1d related files

* bump difftest: use clang to compile verialted files

* dcache: add BankedDataArray

* dcache: fix data read way_en

* dcache: fix banked data wmask

* dcache: replay conflict correctly

When conflict is detected:
* Report replay
* Disable fast wakeup

* dcache: fix bank addr match logic

* dcache: add bank conflict perf counter

* dcache: fix miss perf counters

* chore: make lsq data print perttier

* dcache: enable banked ecc array

* dcache: set dcache size to 128KB

* dcache: read mainpipe data from banked data array

* dcache: add independent mainpipe data read port

* dcache: revert size change

* Size will be changed after main pipe refactor

* Merge remote-tracking branch 'origin/master' into l1-size

* dcache: reduce banked data load conflict

* MainPipe: ReleaseData for all replacement even if it's clean

* dcache: set dcache size to 128KB

BREAKING CHANGE: l2 needed to provide right vaddr index to probe l1,
and it has to help l1 to avoid addr alias problem

* chore: fix merge conflict

* Change L2 to non-inclusive / Add alias bits in L1D

* debug: hard coded dup data array for debuging

* dcache: fix ptag width

* dcache: fix amo main pipe req

* dcache: when probe, use vaddr for main pipe req

* dcache: include vaddr in atomic unit req

* dcache: fix get_tag() function

* dcache: fix writeback paddr

* huancun: bump version

* dcache: erase block offset bits in release addr

* dcache: do not require probe vaddr != 0

* dcache: opt banked data read timing

* bump huancun

* dcache: fix atom unit pipe req vaddr

* dcache: simplify main pipe writeback_vaddr

* bump huancun

* dcache: remove debug data array

* Turn on all usr bits in L1

* Bump huancun

* Bump huancun

* enable L2 prefetcher

* bump huancun

* set non-inclusive L2/L3 + 128KB L1 as default config

* Use data in TLBundleB to hint ProbeAck beeds data

* mmu.l2tlb: mem_resp now fills multi mq pte buffer

mq entries can just deq without accessing l2tlb cache

* dcache: handle dirty userbit

* bump huancun

* chore: l1 cache code clean up

* Remove l1plus cache
* Remove HasBankedDataArrayParameters

* Add bus pmu between L3 and Mem

* bump huncun

* IFU: add performance counters and mmio af

* icache replacement policy moniter

* ifu miss situation moniter

* icache miss rate

* raise access fault when found mmio req

* Add framework for seperated main pipe and reg meta array

* Rewrite miss queue for seperated pipes

* Add RefillPipe

* chore: rename NewSbuffer.scala

* cache: add CacheInstruction opcode and reg list

* CSR: add cache control registers

* Add Replace Pipe

* CacheInstruction: add CSRs for cache instruction

* mem: remove store replay unit

* Perf counter to be added
* Timing opt to be done

* mem: update sbuffer to support new dcache

* sbuffer: fix missqueue time out logic

* Merge remote-tracking branch 'origin/master' into dcache-rm-sru

* chore: fix merge conflict, remove nStoreReplayEntries

* Temporarily disable TLMonitor

* Bump huancun (L2/L3 MSHR bug fix)

* Rewrite main pipe

* ReplacePipe: read meta to decide whether data should be read

* RefillPipe: add a store resp port

* MissQueue: new req should be rejected according to set+way

* Add replacement policy interface

* sbuffer: give missq replay the highest priority

Now we give missqReplayHasTimeOut the highest priority, as eviction
has already happened
Besides, it will fix the problem that fix dcache eviction generate logic
gives the wrong sbuffer id

* Finish DCache framework

* Split meta & tag and use regs to build meta array

* sbuffer: use new dcache io

* dcache: update dcache resp in memblock and fake d$

* Add atomics processing flow

* Refactor Top

* Bump huancun

* DCacheWrapper: disable ld fast wakeup only when bank conflict

* sbuffer: update dcache_resp difftest io

* MainPipe: fix combinational loop

* Sbuffer: fix bug in assert

* RefillPipe: fix bug of getting tag from addr

* dcache: ~0.U should restrict bit-width

* LoadPipe: fix bug in assert

* ReplacePipe: addr to be replaced should be block-aligned

* MainPipe: fix bug in required coh sending to miss queue

* DCacheWrapper: tag write in refill pipe should always be ready

* MainPipe: use replacement way_en when the req is from miss queue

* MissQueue: refill data should be passed on to main pipe

* MainPipe: do not use replacement way when tag match

* CSR: clean up cache op regs

* chore: remove outdated comments

* ReplacePipe: fix stupid bug

* dcache: replace checkOneHot with assert

* alu: fix bug of rev8 & orc.b instruction

* MissQueue: fix bug in the condition of mshr accepting a req

* MissQueue: add perf counters

* chore: delete out-dated code

* chore: add license

* WritebackQueue: distinguish id from miss queue

* AsynchronousMetaArray: fix bug

* Sbuffer: fix difftest io

* DCacheWrapper: duplicate one more tag copy for main pipe

* Add perf cnt to verify whether replacing is too early

* dcache: Release needs to wait for refill pipe

* WritebackQueue: fix accept condition

* MissQueue: remove unnecessary assert

* difftest: let refill check ingore illegal mem access

* Parameters: enlarge WritebackQueue to break dead-lock

* DCacheWrapper: store hit wirte should not be interrupted by refill

* Config: set nReleaseEntries to twice of nMissEntries

* DCacheWrapper: main pipe read should block refill pipe by set

Co-authored-by: William Wang <[email protected]>
Co-authored-by: LinJiawei <[email protected]>
Co-authored-by: TangDan <[email protected]>
Co-authored-by: LinJiawei <[email protected]>
Co-authored-by: ZhangZifei <[email protected]>
Co-authored-by: wangkaifan <[email protected]>
Co-authored-by: JinYue <[email protected]>
Co-authored-by: Zhangfw <[email protected]>

show more ...