Commit graph

101 commits

Author SHA1 Message Date
Laurence Jones
bcce4afe5e
enhance: Flags now superceed all log levels (#3496)
* enhance: Flags now superceed all log levels

* enhance: remove global var for local scope

* test

---------

Co-authored-by: marco <marco@crowdsec.net>
2025-03-07 13:42:08 +00:00
victoredvardsson
466f39b880
Add possibility to configure log format #799 (#2941)
* make it possible to enable json log

* fix

* fix typo

* fix typo

* fix typo

* fix typo

* fix typo

* fix typo

* Add error handling

* Add log_format to default config

* Fix syntax error in if statement

* Fix typo

* Fix typo

* Fix some typos and change naming from native to text, makes more sense

* Set same timestamp format for json logging

* Fix formatting

* Move in if statement under previous

* Fix some formatting that got messed up

* Default to text formatter, if log_format is not configured.

* defining logFormatter outside if statement so that log.SetFormatter(logFormatter) is not undefined when function is called

* Add variables that were undefined

* Argument were missing when calling SetDefaultLoggerConfig function

* Fix order of arguments passed

* Fix order of arguments passed

* Fix typo

* Implicit log_format = "text"

* functional test

* ignore log_format in FatalHook

* make it possible to enable json log

* fix

* fix typo

* fix typo

* fix typo

* fix typo

* fix typo

* fix typo

* Add error handling

* Add log_format to default config

* Fix syntax error in if statement

* Fix typo

* Fix typo

* Fix some typos and change naming from native to text, makes more sense

* Set same timestamp format for json logging

* Fix formatting

* Move in if statement under previous

* Fix some formatting that got messed up

* Default to text formatter, if log_format is not configured.

* defining logFormatter outside if statement so that log.SetFormatter(logFormatter) is not undefined when function is called

* Add variables that were undefined

* Argument were missing when calling SetDefaultLoggerConfig function

* Fix order of arguments passed

* Fix order of arguments passed

* Fix typo

* Implicit log_format = "text"

* functional test

* ignore log_format in FatalHook

* lint

* fix func test

* lint

* remove < > characters from log

---------

Co-authored-by: Victor Edvardsson <victor.edvardsson@loopia.se>
Co-authored-by: marco <marco@crowdsec.net>
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2024-12-23 11:08:33 +01:00
mmetc
4748720a07
refactor pkg/leakybucket (#3371)
* refact pkg/leakybucket - call LoadBuckets with Item instances
* extract compileScopeFilter()
2024-12-20 14:33:24 +01:00
mmetc
26c15a1267
refact pkg/database: clean up code and error messages (#3263)
* refact pkg/database: extract alertfilter.go
* refact pkg/database: extract function rollbackOnError(); dry error messages
2024-12-19 17:01:16 +01:00
mmetc
411bb48a81
loop performance optimizations / 1 (#3313)
* rangeValCopy: each iteration copies 248 bytes
* rangeValCopy: each iteration copies 576 bytes
* rangeValCopy: each iteration copies 376 bytes
* rangeValCopy: each iteration copies 312 bytes
* enable linter: gocritic/rangeValCopy
2024-12-05 18:04:26 +01:00
mmetc
7a1ad8376a
lint: style, autofix (#3354) 2024-12-05 10:40:48 +01:00
he2ss
19b70f10be
add HTTP datasource (#3294) 2024-11-05 14:15:04 +01:00
Thibault "bui" Koechlin
1616991398
Alert context appsec (#3288) 2024-11-04 10:02:55 +01:00
mmetc
bc6be99b97
cscli refact: package clialert, clidecision (#3203)
* cscli refact: package clialert, clidecision

* refact: function SanitizeScope()

* lint
2024-09-03 12:37:38 +02:00
Robert Nagy
27559d6636
implement GetFSType on openbsd with the correct statfs struct member (#3191) 2024-08-27 12:41:53 +02:00
blotus
a3d7900b5f
update expr (#3144) 2024-07-22 12:14:46 +02:00
mmetc
06720538f5
revert "db: round created, updated... timestamps to 1 second" (#3127)
this fixes functional tests with decision stream startup=true
2024-07-12 17:02:44 +02:00
mmetc
206211ce53
lint: import statement order (#3085)
* lint: import statement order

* lint
2024-06-26 12:16:17 +02:00
mmetc
8a259fd25b
lint (copyloopvar) (#2971) 2024-06-20 22:13:26 +02:00
mmetc
659774fd3d
refactor: prefer logrus.WithField over WithFields with a single param (#3087) 2024-06-20 10:38:23 +02:00
mmetc
ae58b158a5
enable linter: revive (var-declaration) (#3069) 2024-06-11 09:26:50 +02:00
mmetc
31ed9fb5ee
enable linter: revive (indent-error-flow) (#3068)
* enable linter: revive (indent-error-flow)

* lint
2024-06-10 17:36:22 +02:00
mmetc
8501f459c3
db,lapi: round durations and timestamps to 1 second (#3015)
* cscli/lapi: round decision duration to 1 second

* db: round created, updated... timestamps to 1 second
2024-05-31 16:24:33 +02:00
mmetc
529d3b2079
minor lint/readability (#2976)
* simplify a couple loops

* if/else -> switch

* drop redundant else

* comment + drop var declaration + explicit zero return

* lint (whitespace/fmt.Errorf)
2024-05-02 13:42:47 +02:00
blotus
ceb4479ec4
add zfs magic for GetFSType (#2950) 2024-04-25 15:05:11 +02:00
mmetc
60431804d8
db config: don't exit setup if can't detect fs, improve detection for freebsd (#2963) 2024-04-25 11:11:57 +02:00
Manuel Sabban
bd785ede15
Fix armhf (#2886)
* armhf compile fix
2024-03-12 17:33:22 +01:00
Manuel Sabban
1a56a0e0b9
armhf fix for getfstype (#2884)
* armhf fix for getfstype
2024-03-12 14:33:10 +01:00
blotus
5731491b4e
Auto detect if reading logs or storing sqlite db on a network share (#2241) 2024-03-07 14:04:50 +01:00
Thibault "bui" Koechlin
717fc97ca0
add SetMeta and SetParsed helpers (#2845)
* add SetMeta and SetParsed helpers
2024-02-14 13:38:40 +01:00
mmetc
f306d59016
logging: full timestamp with timezone in crowdsec.log (#2707)
RFC3339 = "2006-01-02T15:04:05Z07:00" (same as /var/log/syslog)
2024-01-08 21:20:25 +01:00
mmetc
5622ac8338
CI: enable testifylint (#2696)
- reverse actual and expected values
 - use assert.False, assert.True
 - use assert.Len, assert.Emtpy
 - use require.Error, require.NoError
 - use assert.InDelta
2024-01-05 15:26:13 +01:00
AlteredCoder
b1f85693c2
Appsec improvement and fixes after merge (#2645) 2023-12-08 10:25:00 +01:00
Thibault "bui" Koechlin
8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc
ec199162dc
iso8601: use yyyy-mm-dd in log timestamps instead of dd-mm-yyyy (#2564)
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-11-24 14:59:28 +01:00
Thibault "bui" Koechlin
1dcf9d1ae1
Improved expr debugger (#2495)
* new expr debugger

---------

Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
2023-11-24 11:10:54 +01:00
Laurence Jones
19de3a8a77
Runtime whitelist parsing improvement (#2422)
* Improve whitelist parsing

* Split whitelist check into a function tied to whitelist, also since we check node debug we can make a pointer to node containing whitelist

* No point passing clog as an argument since it is just a pointer to node we already know about

* We should break instead of returning false, false as it may have been whitelisted by ips/cidrs

* reimplement early return if expr errors

* Fix lint and dont need to parse ip back to string just loop over sources

* Log error with node logger as it provides context

* Move getsource to a function cleanup some code

* Change func name

* Split out compile to a function so we can use in tests. Add a bunch of tests

* spell correction

* Use node logger so it has context

* alternative solution

* quick fixes

* Use containswls

* Change whitelist test to use parseipsource and only events

* Make it simpler

* Postoverflow tests, some basic ones to make sure it works

* Use official pkg

* Add @mmetc reco

* Add @mmetc reco

* Change if if to a switch to only evaluate once

* simplify assertions

---------

Co-authored-by: bui <thibault@crowdsec.net>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-10-16 10:08:57 +01:00
Thibault "bui" Koechlin
e4dcdd2572
fix include_capi filter (#2478) 2023-09-20 11:56:00 +02:00
mmetc
e36df40ba7
pkg/types cleanup (#2398)
* move function GetLineCountForFile from pkg/types to cscli
* move ParseDuration from pkg/types to pkg/database
* remove unused types.Profile, types.RemediationProfile
2023-08-24 09:44:46 +02:00
mmetc
bd41f855cf
errors.Wrap -> fmt.Errorf (#2317) 2023-06-29 11:34:59 +02:00
mmetc
76429f033a
trim pkg/types: move DataSet/GetData to pkg/cwhub, removed unused Clone function (#2271) 2023-06-08 16:49:51 +02:00
mmetc
cf747d65e0
fix missing import (#2275) 2023-06-08 15:49:37 +02:00
mmetc
25bb23d8b7
minor refactor to pkg/types, cscli machines (#2270)
* cleanup: separate ui and logic
* trim some code from pkg/types
2023-06-08 15:08:51 +02:00
mmetc
6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies (#2269) 2023-06-08 15:07:30 +02:00
mmetc
5b3200173e
don't pre-create log files (not required anymore) (#2267)
The lumberjack package fixed the issue in natefinch/lumberjack#83 (tested with umask 002) and this code is now redundant since we updated the dependency to v2.2.1.
2023-06-07 12:58:35 +02:00
mmetc
396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap (#2235)
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
mmetc
92a9d6c321
types.InSlice() -> slices.Contains() (#2246) 2023-05-31 12:39:22 +02:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216)
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
blotus
772d5b5c32
Add experimental support for re2 (#2138) 2023-03-28 16:26:47 +02:00
Thibault "bui" Koechlin
618be9ff68
properly update the time structure within event (#2122)
* properly update the time structure within event to ensure it works in time-machine

* move LIVE and TIMEMACHINE to pkg/types : less code needs to import leakybucket package, and we avoid duplicating constants
2023-03-16 16:25:50 +01:00
Thibault "bui" Koechlin
5b0fe4b7f1
support for regexps result cache (#2104)
* support for regexps result cache : gcache + xxhash

Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-03-08 16:07:49 +01:00
blotus
85ab9c68a2
Add cscli papi status and cscli papi sync (#2091) 2023-03-03 13:46:28 +01:00
Thibault "bui" Koechlin
a0b264047c
allow user to specify stash's cache strategy (#2037) 2023-02-06 15:42:55 +01:00
Cristian Nitescu
987f119c4b
v3 capi and blocklists links support (#2019)
* v3 model generation

* v3 model generation

* comms

* fixes after master merge

* missing reader close

* use constants defined for types

---------

Co-authored-by: bui <thibault@crowdsec.net>
2023-02-06 14:06:14 +01:00
AlteredCoder
7e871d2278
rename PAPI base URL (#2033) 2023-02-03 12:10:02 +01:00