Use static linking for libpcap and fix error with truncated packets

Fix license file
Update java middleware to not use apache commons
2026-10-19 16:15:42 +00:00 · 2016-05-03 17:55:38 +05:00 · 2016-05-03 10:28:15 +05:00 · 2016-05-02 16:31:38 +05:00 · 2016-04-30 18:35:51 +05:00 · 2016-04-30 18:35:13 +05:00
26 changed files with 884 additions and 204 deletions
@@ -11,4 +11,7 @@

 *.test
 .idea
+*.iml
 gor
+
+*.mprof
@@ -1,3 +1,6 @@
 language: go
 go: 1.6
-script: sudo -E bash -c "source /etc/profile && eval '$(gimme 1.6)' && export GOPATH=$HOME/gopath:$GOPATH && go get && GORACE='halt_on_error=1' go test ./... -v -timeout 60s -race"
+script: sudo -E bash -c "source /etc/profile && eval '$(gimme 1.6)' && export GOPATH=$HOME/gopath:$GOPATH && go get && GORACE='halt_on_error=1' go test ./... -v -timeout 120s -race"
+
+before_install:
+  - sudo apt-get install libpcap-dev -y
@@ -2,7 +2,7 @@ END-USER LICENSE AGREEMENT

 ------------------------------------------------------------------------------

-IMPORTANT: THIS SOFTWARE END-USER LICENSE AGREEMENT ("EULA") IS A LEGAL AGREEMENT (“Agreement”) BETWEEN YOU (THE CUSTOMER, EITHER AS AN INDIVIDUAL OR, IF PURCHASED OR OTHERWISE ACQUIRED BY OR FOR AN ENTITY, AS AN ENTITY) AND CONTRIBUTED SYSTEMS. READ IT CAREFULLY BEFORE COMPLETING THE INSTALLATION PROCESS AND USING GOR PRO AND RELATED SOFTWARE COMPONENTS (“SOFTWARE”). IT PROVIDES A LICENSE TO USE THE SOFTWARE AND CONTAINS WARRANTY INFORMATION AND LIABILITY DISCLAIMERS. BY INSTALLING AND USING THE SOFTWARE, YOU ARE CONFIRMING YOUR ACCEPTANCE OF THE SOFTWARE AND AGREEING TO BECOME BOUND BY THE TERMS OF THIS AGREEMENT.
+IMPORTANT: THIS SOFTWARE END-USER LICENSE AGREEMENT ("EULA") IS A LEGAL AGREEMENT (“Agreement”) BETWEEN YOU (THE CUSTOMER, EITHER AS AN INDIVIDUAL OR, IF PURCHASED OR OTHERWISE ACQUIRED BY OR FOR AN ENTITY, AS AN ENTITY) AND REPLAY SOFTWARE. READ IT CAREFULLY BEFORE COMPLETING THE INSTALLATION PROCESS AND USING GOR PRO AND RELATED SOFTWARE COMPONENTS (“SOFTWARE”). IT PROVIDES A LICENSE TO USE THE SOFTWARE AND CONTAINS WARRANTY INFORMATION AND LIABILITY DISCLAIMERS. BY INSTALLING AND USING THE SOFTWARE, YOU ARE CONFIRMING YOUR ACCEPTANCE OF THE SOFTWARE AND AGREEING TO BECOME BOUND BY THE TERMS OF THIS AGREEMENT.

 ------------------------------------------------------------------------------

@@ -14,19 +14,19 @@ In order to use the Software under this Agreement, you must receive a “Source

 1.2 Unlimited Organization License. If you purchased an Organization License (included with the Gor Pro Software), you may install the Software on an unlimited number of Hosts. “Host” means any physical or virtual machine which is controlled by you. You may also run an unlimited number of Workers. “Worker” means a thread within a Gor server process which executes jobs. You may concurrently run the software on an unlimited number of Hosts, with each host running an unlimited number of Workers.

-1.3 Limited Organization License. If you purchased an Organization License (included with the Gor Enterprise Software), you may install the Software on an unlimited number of Hosts. “Host” means any physical or virtual machine which is controlled by you. The aggregate number of Workers run by the hosts must not exceed the maximum number of Workers authorized at the time of purchase. “Worker” means a running Gor instance which intercept or replay traffic. In order to run additional Workers, you must purchase an additional allowance from Contributed Systems.
+1.3 Limited Organization License. If you purchased an Organization License (included with the Gor Enterprise Software), you may install the Software on an unlimited number of Hosts. “Host” means any physical or virtual machine which is controlled by you. The aggregate number of Workers run by the hosts must not exceed the maximum number of Workers authorized at the time of purchase. “Worker” means a running Gor instance which intercept or replay traffic. In order to run additional Workers, you must purchase an additional allowance from Replay Software.

-1.4 Appliance License. If you purchased an Appliance License, you may distribute the Software in any applications, frameworks, or elements (collectively referred to as an “Application” or “Applications”) that you develop using the Software in accordance with this EULA, provided that such distribution does not violate the restrictions set forth in section 3 of this EULA. You must not remove, obscure or interfere with any copyright, acknowledgment, attribution, trademark, warning or disclaimer statement affixed to, incorporated in or otherwise applied in connection with the Software. You are required to ensure that the Software is not reused by or with any applications other than those with which you distribute it as permitted herein. For example, if You install the Software on a customer’s server, that customer is not permitted to use the Software independently of your Application. You must inform Contributed Systems of your knowledge of any infringing use of the Software by any of your customers. You are liable for compliance by those third parties with the terms and conditions of this EULA. You will not owe Contributed Systems any royalties for your distribution of the Software in accordance with this EULA.
+1.4 Appliance License. If you purchased an Appliance License, you may distribute the Software in any applications, frameworks, or elements (collectively referred to as an “Application” or “Applications”) that you develop using the Software in accordance with this EULA, provided that such distribution does not violate the restrictions set forth in section 3 of this EULA. You must not remove, obscure or interfere with any copyright, acknowledgment, attribution, trademark, warning or disclaimer statement affixed to, incorporated in or otherwise applied in connection with the Software. You are required to ensure that the Software is not reused by or with any applications other than those with which you distribute it as permitted herein. For example, if You install the Software on a customer’s server, that customer is not permitted to use the Software independently of your Application. You must inform Replay Software of your knowledge of any infringing use of the Software by any of your customers. You are liable for compliance by those third parties with the terms and conditions of this EULA. You will not owe Replay Software any royalties for your distribution of the Software in accordance with this EULA.

 1.5 Archive Copies. You are entitled to make a reasonable amount of copies of the Software for archival purposes. Each copy must reproduce all copyright and other proprietary rights notices on or in the Software Product.

 1.6 Electronic Delivery. All Software and license documentation shall be delivered by electronic means unless otherwise specified on the applicable invoice or at the time of purchase. Software shall be deemed delivered when it is made available for download by you (“Delivery”).        

-2. Modifications. Contributed Systems shall provide you with source code so that you can create Modifications of the original software. “Modification” means: (a) any addition to or deletion from the contents of a file included in the original Software or previous Modifications created by You, or (b) any new file that contains any part of the original Software or previous Modifications. While you retain all rights to any original work authored by you as part of the Modifications, We continue to own all copyright and other intellectual property rights in the Software.
+2. Modifications. Replay Software shall provide you with source code so that you can create Modifications of the original software. “Modification” means: (a) any addition to or deletion from the contents of a file included in the original Software or previous Modifications created by You, or (b) any new file that contains any part of the original Software or previous Modifications. While you retain all rights to any original work authored by you as part of the Modifications, We continue to own all copyright and other intellectual property rights in the Software.

 3. Restricted Uses. 

-3.1 You shall not (and shall not allow any third party to): (a) decompile, disassemble, or otherwise reverse engineer the Software or attempt to reconstruct or discover any source code, underlying ideas, algorithms, file formats or programming interfaces of the Software by any means whatsoever (except and only to the extent that applicable law prohibits or restricts reverse engineering restrictions); (b) distribute, sell, sublicense, rent, lease or use the Software for time sharing, hosting, service provider or like purposes, except as expressly permitted under this Agreement; (c) redistribute the Software or Modifications other than by including the Software or a portion thereof within your own product, which must have substantially different functionality than the Software or Modifications and must not allow any third party to use the Software or Modifications, or any portions thereof, for software development or application development purposes; (d) redistribute the Software as part of a product, "appliance" or "virtual server"; (e) redistribute the Software on any server which is not directly under your control; (f) remove any product identification, proprietary, copyright or other notices contained in the Software; (g) modify any part of the Software, create a derivative work of any part of the Software (except as permitted in Section 4), or incorporate the Software, except to the extent expressly authorized in writing by Contributed Systems; (h) publicly disseminate performance information or analysis (including, without limitation, benchmarks) from any source relating to the Software; (i) utilize any equipment, device, software, or other means designed to circumvent or remove any form of Source URL or copy protection used by Contributed Systems in connection with the Software, or use the Software together with any authorization code, Source URL, serial number, or other copy protection device not supplied by Contributed Systems; (j) use the Software to develop a product which is competitive with any Contributed Systems product offerings; or (k) use unauthorized Source URLS or keycode(s) or distribute or publish Source URLs or keycode(s), except as may be expressly permitted by Contributed Systems in writing. If your unique Source URL is ever published, Contributed Systems reserves the right to terminate your access without notice.
+3.1 You shall not (and shall not allow any third party to): (a) decompile, disassemble, or otherwise reverse engineer the Software or attempt to reconstruct or discover any source code, underlying ideas, algorithms, file formats or programming interfaces of the Software by any means whatsoever (except and only to the extent that applicable law prohibits or restricts reverse engineering restrictions); (b) distribute, sell, sublicense, rent, lease or use the Software for time sharing, hosting, service provider or like purposes, except as expressly permitted under this Agreement; (c) redistribute the Software or Modifications other than by including the Software or a portion thereof within your own product, which must have substantially different functionality than the Software or Modifications and must not allow any third party to use the Software or Modifications, or any portions thereof, for software development or application development purposes; (d) redistribute the Software as part of a product, "appliance" or "virtual server"; (e) redistribute the Software on any server which is not directly under your control; (f) remove any product identification, proprietary, copyright or other notices contained in the Software; (g) modify any part of the Software, create a derivative work of any part of the Software (except as permitted in Section 4), or incorporate the Software, except to the extent expressly authorized in writing by Replay Software; (h) publicly disseminate performance information or analysis (including, without limitation, benchmarks) from any source relating to the Software; (i) utilize any equipment, device, software, or other means designed to circumvent or remove any form of Source URL or copy protection used by Replay Software in connection with the Software, or use the Software together with any authorization code, Source URL, serial number, or other copy protection device not supplied by Replay Software; (j) use the Software to develop a product which is competitive with any Replay Software product offerings; or (k) use unauthorized Source URLS or keycode(s) or distribute or publish Source URLs or keycode(s), except as may be expressly permitted by Replay Software in writing. If your unique Source URL is ever published, Replay Software reserves the right to terminate your access without notice.

 3.2 UNDER NO CIRCUMSTANCES MAY YOU USE THE SOFTWARE AS PART OF A PRODUCT OR SERVICE THAT PROVIDES SIMILAR FUNCTIONALITY TO THE SOFTWARE ITSELF.

@@ -34,34 +34,34 @@ The Open Source version of the Software (“LGPL Version”) is licensed
 under the terms of the GNU Lesser General Public License versions 3.0
 (“LGPL”) and not under this EULA. 

-4. Ownership. Notwithstanding anything to the contrary contained herein, except for the limited license rights expressly provided herein, Contributed Systems and its suppliers have and will retain all rights, title and interest (including, without limitation, all patent, copyright, trademark, trade secret and other intellectual property rights) in and to the Software and all copies, modifications and derivative works thereof (including any changes which incorporate any of your ideas, feedback or suggestions). You acknowledge that you are obtaining only a limited license right to the Software, and that irrespective of any use of the words “purchase”, “sale” or like terms hereunder no ownership rights are being conveyed to you under this Agreement or otherwise. 
+4. Ownership. Notwithstanding anything to the contrary contained herein, except for the limited license rights expressly provided herein, Replay Software and its suppliers have and will retain all rights, title and interest (including, without limitation, all patent, copyright, trademark, trade secret and other intellectual property rights) in and to the Software and all copies, modifications and derivative works thereof (including any changes which incorporate any of your ideas, feedback or suggestions). You acknowledge that you are obtaining only a limited license right to the Software, and that irrespective of any use of the words “purchase”, “sale” or like terms hereunder no ownership rights are being conveyed to you under this Agreement or otherwise. 

-5. Fees and Payment. The Software license fees will be due and payable in full as set forth in the applicable invoice or at the time of purchase. If the Software does not function properly within two weeks of purchase, please contact us within those two weeks for a refund. You shall be responsible for all taxes, withholdings, duties and levies arising from the order (excluding taxes based on the net income of Contributed Systems). 
+5. Fees and Payment. The Software license fees will be due and payable in full as set forth in the applicable invoice or at the time of purchase. If the Software does not function properly within two weeks of purchase, please contact us within those two weeks for a refund. You shall be responsible for all taxes, withholdings, duties and levies arising from the order (excluding taxes based on the net income of Replay Software). 

 6. Support, Maintenance and Services. Subject to the terms and conditions of this Agreement, as set forth in your invoice, and as set forth on the Gor Pro support page (https://github.com/buger/gor/wiki/Pro-Support), support and maintenance services may be included with the purchase of your license subscription.

 7. Term of Agreement. 

-7.1 Term. This Agreement is effective as of the Delivery of the Software and expires at such time as all license and service subscriptions hereunder have expired in accordance with their own terms (the “Term”). For clarification, the term of your license under this Agreement may be perpetual, limited for Evaluation Version, or designated as a fixed-term license in the Invoice, and shall be specified at your time of purchase. Either party may terminate this Agreement (including all related Invoices) if the other party: (a) fails to cure any material breach of this Agreement within thirty (30) days after written notice of such breach, provided that Contributed Systems may terminate this Agreement immediately upon any breach of Section 3 or if you exceed any other restrictions contained in Section 1, unless otherwise specified in this agreement; (b) ceases operation without a successor; or (c) seeks protection under any bankruptcy, receivership, trust deed, creditors arrangement, composition or comparable proceeding, or if any such proceeding is instituted against such party (and not dismissed within sixty (60) days)). Termination is not an exclusive remedy and the exercise by either party of any remedy under this Agreement will be without prejudice to any other remedies it may have under this Agreement, by law, or otherwise.        
+7.1 Term. This Agreement is effective as of the Delivery of the Software and expires at such time as all license and service subscriptions hereunder have expired in accordance with their own terms (the “Term”). For clarification, the term of your license under this Agreement may be perpetual, limited for Evaluation Version, or designated as a fixed-term license in the Invoice, and shall be specified at your time of purchase. Either party may terminate this Agreement (including all related Invoices) if the other party: (a) fails to cure any material breach of this Agreement within thirty (30) days after written notice of such breach, provided that Replay Software may terminate this Agreement immediately upon any breach of Section 3 or if you exceed any other restrictions contained in Section 1, unless otherwise specified in this agreement; (b) ceases operation without a successor; or (c) seeks protection under any bankruptcy, receivership, trust deed, creditors arrangement, composition or comparable proceeding, or if any such proceeding is instituted against such party (and not dismissed within sixty (60) days)). Termination is not an exclusive remedy and the exercise by either party of any remedy under this Agreement will be without prejudice to any other remedies it may have under this Agreement, by law, or otherwise.        

 7.2 Termination. Upon any termination of this Agreement, you shall cease any and all use of any Software and destroy all copies thereof. 

-7.3 Expiration of License. Upon the expiration of any term under this Agreement, (a) all Software updates and services pursuant to the license shall cease, (b) you may only continue to run existing installations of the Software, (c) you may not install the Software on any additional Hosts, and (d) any new installation of the Software shall require the purchase of a new license subscription from Contributed Systems.
+7.3 Expiration of License. Upon the expiration of any term under this Agreement, (a) all Software updates and services pursuant to the license shall cease, (b) you may only continue to run existing installations of the Software, (c) you may not install the Software on any additional Hosts, and (d) any new installation of the Software shall require the purchase of a new license subscription from Replay Software.

-8. Disclaimer of Warranties. The Software is provided "as is," with all faults, defects and errors, and without warranty of any kind. Contributed Systems does not warrant that the Software will be free of bugs, errors, viruses or other defects, and Contributed Systems shall have no liability of any kind for the use of or inability to use the Software, the Software content or any associated service, and you acknowledge that it is not technically practicable for Contributed Systems to do so. 
-To the maximum extent permitted by applicable law, Contributed Systems disclaims all warranties, express, implied, arising by law or otherwise, regarding the Software, the Software content and their respective performance or suitability for your intended use, including without limitation any implied warranty of merchantability, fitness for a particular purpose.
+8. Disclaimer of Warranties. The Software is provided "as is," with all faults, defects and errors, and without warranty of any kind. Replay Software does not warrant that the Software will be free of bugs, errors, viruses or other defects, and Replay Software shall have no liability of any kind for the use of or inability to use the Software, the Software content or any associated service, and you acknowledge that it is not technically practicable for Replay Software to do so. 
+To the maximum extent permitted by applicable law, Replay Software disclaims all warranties, express, implied, arising by law or otherwise, regarding the Software, the Software content and their respective performance or suitability for your intended use, including without limitation any implied warranty of merchantability, fitness for a particular purpose.

 9. Limitation of Liability. 

-In no event will Contributed Systems be liable for any direct, indirect, consequential, incidental, special, exemplary, or punitive damages or liabilities whatsoever arising from or relating to the Software, the Software content or this Agreement, whether based on contract, tort (including negligence), strict liability or other theory, even if Contributed Systems has been advised of the possibility of such damages.
+In no event will Replay Software be liable for any direct, indirect, consequential, incidental, special, exemplary, or punitive damages or liabilities whatsoever arising from or relating to the Software, the Software content or this Agreement, whether based on contract, tort (including negligence), strict liability or other theory, even if Replay Software has been advised of the possibility of such damages.

-In no event will Contributed Systems' liability exceed the Software license price as indicated in the invoice. The existence of more than one claim will not enlarge or extend this limit.
+In no event will Replay Software' liability exceed the Software license price as indicated in the invoice. The existence of more than one claim will not enlarge or extend this limit.

-10. Remedies. Your exclusive remedy and Contributed Systems’ entire liability for breach of this Agreement shall be limited, at Contributed Systems’ sole and exclusive discretion, to (a) replacement of any defective software or documentation; or (b) refund of the license fee paid to Contributed Systems, payable in accordance with Contributed Systems' refund policy.
+10. Remedies. Your exclusive remedy and Replay Software’ entire liability for breach of this Agreement shall be limited, at Replay Software’ sole and exclusive discretion, to (a) replacement of any defective software or documentation; or (b) refund of the license fee paid to Replay Software, payable in accordance with Replay Software' refund policy.

 11. Acknowledgements.

-11.1 Consent to the Use of Data. You agree that Contributed Systems and its affiliates may collect and use technical information gathered as part of the product support services. Contributed Systems may use this information solely to improve products and services and will not disclose this information in a form that personally identifies you.
+11.1 Consent to the Use of Data. You agree that Replay Software and its affiliates may collect and use technical information gathered as part of the product support services. Replay Software may use this information solely to improve products and services and will not disclose this information in a form that personally identifies you.

 11.2 Verification. We or a certified auditor acting on our behalf, may, upon its reasonable request and at its expense, audit you with respect to the use of the Software. Such audit may be conducted by mail, electronic means or through an in-person visit to your place of business. Any such in-person audit shall be conducted during regular business hours at your facilities and shall not unreasonably interfere with your business activities. We shall not remove, copy, or redistribute any electronic material during the course of an audit. If an audit reveals that you are using the Software in a way that is in material violation of the terms of the EULA, then you shall pay our reasonable costs of conducting the audit. In the case of a material violation, you agree to pay Us any amounts owing that are attributable to the unauthorized use. In the alternative, We reserve the right, at our sole option, to terminate the licenses for the Software.

@@ -74,13 +74,13 @@ In no event will Contributed Systems' liability exceed the Software license pric

 13.1 Entire Agreement. This Agreement sets forth our entire agreement with respect to the Software and the subject matter hereof and supersedes all prior and contemporaneous understandings and agreements whether written or oral.

-13.2 Amendment. Contributed Systems reserves the right, in its sole discretion, to amend this Agreement from time. Amendments to this Agreement can be located at: https://github.com/buger/gor/blob/master/COMM-LICENSE.
+13.2 Amendment. Replay Software reserves the right, in its sole discretion, to amend this Agreement from time. Amendments to this Agreement can be located at: https://github.com/buger/gor/blob/master/COMM-LICENSE.

-13.3 Assignment. You may not assign this Agreement or any of its rights under this Agreement without the prior written consent of Contributed Systems and any attempted assignment without such consent shall be void.
+13.3 Assignment. You may not assign this Agreement or any of its rights under this Agreement without the prior written consent of Replay Software and any attempted assignment without such consent shall be void.

 13.4 Export Compliance. You agree to comply with all applicable laws and regulations, including laws, regulations, orders or other restrictions on export, re-export or redistribution of software.

-13.5 Indemnification. You agree to defend, indemnify, and hold harmless Contributed Systems from and against any lawsuits, claims, losses, damages, fines and expenses (including attorneys' fees and costs) arising out of your use of the Software or breach of this Agreement.
+13.5 Indemnification. You agree to defend, indemnify, and hold harmless Replay Software from and against any lawsuits, claims, losses, damages, fines and expenses (including attorneys' fees and costs) arising out of your use of the Software or breach of this Agreement.

 13.6 Governing Law. This Agreement is governed by the laws of the State of Oregon and the United States without regard to conflicts of laws provisions thereof, and without regard to the United Nations Convention on the International Sale of Goods or the Uniform Computer Information Transactions Act, as currently enacted by any jurisdiction or as may be codified or amended from time to time by any jurisdiction. The jurisdiction and venue for actions related to the subject matter hereof shall be the state of Oregon and United States federal courts located in Portland, Oregon, and both parties hereby submit to the personal jurisdiction of such courts. 

@@ -92,4 +92,4 @@ In no event will Contributed Systems' liability exceed the Software license pric

 13.10 Headings. The headings of sections and paragraphs of this Agreement are for convenience of reference only and are not intended to restrict, affect or be of any weight in the interpretation or construction of the provisions of such sections or paragraphs.

-14. Contact Information. If you have any questions about this EULA, or if you want to contact Contributed Systems for any reason, please direct correspondence to leonsbox@gmail.com
+14. Contact Information. If you have any questions about this EULA, or if you want to contact Replay Software for any reason, please direct correspondence to info@gortool.com
@@ -10,13 +10,12 @@ RUN apt-get update -y
 RUN echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections
 RUN apt-get install oracle-java8-installer -y

-RUN wget http://apache-mirror.rbc.ru/pub/apache//commons/io/binaries/commons-io-2.4-bin.tar.gz -P /tmp
-RUN tar xzf /tmp/commons-io-2.4-bin.tar.gz -C /tmp
+RUN apt-get install libpcap-dev -y
+RUN go get github.com/google/gopacket
+RUN go get -u github.com/golang/lint/golint

 WORKDIR /go/src/github.com/buger/gor/
 ADD . /go/src/github.com/buger/gor/

 RUN javac -cp /tmp/commons-io-2.4/commons-io-2.4.jar ./examples/middleware/echo.java
-
-RUN go get -u github.com/golang/lint/golint
 RUN go get
@@ -1,9 +1,7 @@
-Copyright (c) Contributed Systems LLC
-
-Sidekiq is an Open Source project licensed under the terms of
+Gor is an Open Source project licensed under the terms of
 the LGPLv3 license.  Please see <http://www.gnu.org/licenses/lgpl-3.0.html>
 for license text.

 Gor Pro has a commercial-friendly license allowing private forks
-and modifications of Gor.  Please see http://gortool.com/pro/ for
+and modifications of Gor.  Please see http://gortool.com/#pro for
 more detail.  You can find the commercial license terms in COMM-LICENSE.
@@ -1,14 +1,15 @@
 SOURCE = emitter.go gor.go gor_stat.go input_dummy.go input_file.go input_raw.go input_tcp.go limiter.go output_dummy.go output_file.go input_http.go output_http.go output_tcp.go plugins.go settings.go test_input.go elasticsearch.go http_modifier.go http_modifier_settings.go http_client.go middleware.go protocol.go
 SOURCE_PATH = /go/src/github.com/buger/gor/
 RUN = docker run -v `pwd`:$(SOURCE_PATH) -p 0.0.0.0:8000:8000 -t -i gor
+BENCHMARK = BenchmarkRAWInput

-release: release-x86 release-x64
+release: release-x64

 release-x64:
-	docker run -v `pwd`:$(SOURCE_PATH) -t --env GOOS=linux --env GOARCH=amd64 --env CGO_ENABLED=0 -i gor go build -ldflags "-X main.VERSION=$(VERSION)"&& tar -czf gor_$(VERSION)_x64.tar.gz gor && rm gor
+	docker run -v `pwd`:$(SOURCE_PATH) -t --env GOOS=linux --env GOARCH=amd64  -i gor go build -ldflags "-X main.VERSION=$(VERSION) -extldflags "-static""&& tar -czf gor_$(VERSION)_x64.tar.gz gor && rm gor

 release-x86:
-	docker run -v `pwd`:$(SOURCE_PATH) -t --env GOOS=linux --env GOARCH=386 --env CGO_ENABLED=0 -i gor go build -ldflags "-X main.VERSION=$(VERSION)" && tar -czf gor_$(VERSION)_x86.tar.gz gor && rm gor
+	docker run -v `pwd`:$(SOURCE_PATH) -t --env GOOS=linux --env GOARCH=386 -i gor go build -ldflags "-X main.VERSION=$(VERSION)" && tar -czf gor_$(VERSION)_x86.tar.gz gor && rm gor

 build:
 	docker build -t gor .
@@ -24,7 +25,10 @@ race:
 	$(RUN) go test ./... $(ARGS) -v -race -timeout 15s

 test:
-	$(RUN) go test ./... -timeout 10s $(ARGS) -v
+	$(RUN) go test ./. -timeout 30s $(ARGS) -v
+
+test_all:
+	$(RUN) go test ./... -timeout 30s $(ARGS) -v

 testone:
 	$(RUN) go test ./... -timeout 4s -run $(TEST) $(ARGS) -v
@@ -40,7 +44,12 @@ vet:
 	$(RUN) go vet

 bench:
-	$(RUN) go test -v -run NOT_EXISTING -bench HTTP
+	$(RUN) go test -v -run NOT_EXISTING -bench $(BENCHMARK) -benchtime 5s
+
+profile_test:
+	$(RUN) go test $(LDFLAGS) -run NOT_EXISTING -test.benchmem -bench $(BENCHMARK) ./. $(ARGS) -benchtime 5s -memprofile mem.mprof -v
+	$(RUN) go test $(LDFLAGS) -run NOT_EXISTING -test.benchmem -bench $(BENCHMARK) ./. $(ARGS) -benchtime 5s -cpuprofile cpu.out -v
+	$(RUN) go test $(LDFLAGS) -run NOT_EXISTING -test.benchmem -bench $(BENCHMARK) ./. $(ARGS) -c

 # Used mainly for debugging, because docker container do not have access to parent machine ports
 run:
@@ -1,4 +1,4 @@
-[![GitHub release](https://img.shields.io/github/release/buger/gor.svg?maxAge=2592000)]() [![Build Status](https://travis-ci.org/buger/gor.png?branch=master)](https://travis-ci.org/buger/gor) [![codebeat](https://codebeat.co/badges/6427d589-a78e-416c-a546-d299b4089893)](https://codebeat.co/projects/github-com-buger-gor) [![Go Report Card](https://goreportcard.com/badge/github.com/buger/gor)](https://goreportcard.com/report/github.com/buger/gor) [![Join the chat at https://gitter.im/buger/gor](https://badges.gitter.im/buger/gor.svg)](https://gitter.im/buger/gor?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
+[![GitHub release](https://img.shields.io/github/release/buger/gor.svg?maxAge=3600)](https://github.com/buger/gor/releases) [![codebeat](https://codebeat.co/badges/6427d589-a78e-416c-a546-d299b4089893)](https://codebeat.co/projects/github-com-buger-gor) [![Go Report Card](https://goreportcard.com/badge/github.com/buger/gor)](https://goreportcard.com/report/github.com/buger/gor) [![Join the chat at https://gitter.im/buger/gor](https://badges.gitter.im/buger/gor.svg)](https://gitter.im/buger/gor?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

 ## About

@@ -16,7 +16,7 @@ Here is basic workflow: The listener server catches http traffic and sends it to

 ## Want to Upgrade?

-I also sell Gor Pro, extensions to Gor which provide more features, a commercial-friendly license and allow you to support high quality open source development all at the same time. Please see the Gor [homepage](http://gortool.com) for more detail.
+I also sell Gor Pro, extensions to Gor which provide more features, a commercial-friendly license and allow you to support high quality open source development all at the same time. Please see the Gor [homepage](https://gortool.com/) for more detail.

 Subscribe to the [quarterly newsletter](https://tinyletter.com/gor) to stay informed about the latest features and changes to Gor and its bigger siblings.

@@ -41,15 +41,6 @@ It's recommended to use separate server for replaying traffic, but if you have e
 sudo gor --input-raw :80 --output-http "http://staging.com"
 ```

-### Guarantee of replay and HTTP input
-Due to how traffic interception works, there is chance of missing requests. If you want guarantee that requests will be replayed you can use http input, but it will require changes in your app as well. 
-
-```
-sudo gor --input-http :28019 --output-http "http://staging.com"
-```
-
-Then in your application you should send copy (e.g. like reverse proxy) all incoming requests to Gor http input. 
-
 ## Configuration

 ### Forward to multiple addresses
@@ -282,6 +273,13 @@ gor --input-raw :80 --output-http "http://user:pass@staging .com"

 Note: This will overwrite any Authorization headers in the original request.

+### Traffic interception engine
+By default Gor use `libpcap` for intercepting traffic. If you have any troubles with it, you may try alternative engine: `raw_socket`.
+
+```
+sudo gor --input-raw :80 --input-raw-engine "libpcap" --output-http "http://staging.com"
+```
+

 ## Stats 

@@ -0,0 +1,7 @@
+dependencies:
+  pre:
+    - sudo apt-get install libpcap-dev -y
+
+test:
+  override:
+    - sudo bash -l -c "export GOPATH='/home/ubuntu/.go_workspace:/usr/local/go_workspace:/home/ubuntu/.go_project' && GORACE='halt_on_error=1' /usr/local/go/bin/go test ./... -v -timeout 120s -race"
@@ -2,8 +2,6 @@ import java.io.BufferedReader;
 import java.io.IOException;
 import java.io.InputStreamReader;

-import org.apache.commons.io.IOUtils;
-
 public class echo {
    public static void main(String[] args) {
        if(args != null){
@@ -24,7 +22,6 @@ public class echo {

            }
        } catch (IOException e) {
-            IOUtils.closeQuietly(stdin);
        }
    }
 }
@@ -45,7 +45,7 @@ func main() {
 		buf := make([]byte, len(encoded)/2)
 		hex.Decode(buf, encoded)

-		go process(buf)
+		process(buf)
 	}
 }

@@ -8,7 +8,9 @@ import (
 	"net"
 	"net/url"
 	"runtime/debug"
+	"strconv"
 	"strings"
+	"syscall"
 	"time"
 )

@@ -101,7 +103,14 @@ func (c *HTTPClient) isAlive() bool {

 	// Ready 1 byte from socket without timeout to check if it not closed
 	c.conn.SetReadDeadline(time.Now().Add(time.Millisecond))
-	if _, err := c.conn.Read(one); err == io.EOF {
+	_, err := c.conn.Read(one)
+
+	if err == nil {
+		return true
+	} else if err == io.EOF {
+		return false
+	} else if err == syscall.EPIPE {
+		Debug("Detected broken pipe.", err)
 		return false
 	}

@@ -202,6 +211,14 @@ func (c *HTTPClient) Get(path string) (response []byte, err error) {
 	return c.Send([]byte(payload))
 }

+func (c *HTTPClient) Post(path string, body []byte) (response []byte, err error) {
+	payload := "POST " + path + " HTTP/1.1\r\n"
+	payload += "Content-Length: " + strconv.Itoa(len(body)) + "\r\n\r\n"
+	payload += string(body)
+
+	return c.Send([]byte(payload))
+}
+
 const (
 	// https://support.cloudflare.com/hc/en-us/articles/200171936-Error-520-Web-server-is-returning-an-unknown-error
 	HTTP_UNKNOWN_ERROR = "520"
@@ -2,6 +2,8 @@ package main

 import (
 	"bufio"
+	"bytes"
+	"io"
 	"log"
 	"os"
 	"strconv"
@@ -53,36 +55,53 @@ func (i *FileInput) String() string {
 func (i *FileInput) emit() {
 	var lastTime int64

-	// reader := bufio.NewReader(conn)
-	scanner := bufio.NewScanner(i.file)
-	scanner.Split(payloadScanner)
+	payloadSeparatorAsBytes := []byte(payloadSeparator)
+	reader := bufio.NewReader(i.file)
+	var buffer bytes.Buffer

-	for scanner.Scan() {
-		buf := scanner.Bytes()
-		meta := payloadMeta(buf)
+	for {
+		line, err := reader.ReadBytes('\n')

-		if len(meta) > 2 && meta[0][0] == RequestPayload {
-			ts, _ := strconv.ParseInt(string(meta[2]), 10, 64)
-
-			if lastTime != 0 {
-				timeDiff := ts - lastTime
-
-				if i.speedFactor != 1 {
-					timeDiff = int64(float64(timeDiff) / i.speedFactor)
-				}
-
-				time.Sleep(time.Duration(timeDiff))
+		if err != nil {
+			if err != io.EOF {
+				log.Fatal(err)
 			}
+			break

-			lastTime = ts
 		}

-		// scanner returs only pointer, so to remove data-race we have to allocate new array
-		newBuf := make([]byte, len(buf))
-		copy(newBuf, buf)
+		if bytes.Equal(payloadSeparatorAsBytes[1:], line) {
+			asBytes := buffer.Bytes()[:buffer.Len()-1]
+			buffer.Reset()
+
+			meta := payloadMeta(asBytes)
+
+			if len(meta) > 2 && meta[0][0] == RequestPayload {
+				ts, _ := strconv.ParseInt(string(meta[2]), 10, 64)
+
+				if lastTime != 0 {
+					timeDiff := ts - lastTime
+
+					if i.speedFactor != 1 {
+						timeDiff = int64(float64(timeDiff) / i.speedFactor)
+					}
+
+					time.Sleep(time.Duration(timeDiff))
+				}
+
+				lastTime = ts
+			}
+
+			// Bytes() returns only pointer, so to remove data-race copy the data to an array
+			newBuf := make([]byte, len(asBytes))
+			copy(newBuf, asBytes)
+
+			i.data <- newBuf
+		} else {
+			buffer.Write(line)
+		}

-		i.data <- newBuf
 	}

-	log.Println("FileInput: end of file")
+	log.Printf("FileInput: end of file '%s'\n", i.path)
 }
@@ -0,0 +1,220 @@
+package main
+
+import (
+	"bytes"
+	"errors"
+	"io"
+	"io/ioutil"
+	"os"
+	"sync"
+	"syscall"
+	"testing"
+	"time"
+)
+
+func TestInputFileWithGET(t *testing.T) {
+
+	input := NewTestInput()
+	rg := NewRequestGenerator([]io.Reader{input}, func() { input.EmitGET() }, 1)
+	readPayloads := [][]byte{}
+
+	// Given a capture file with a GET request
+	expectedCaptureFile := CreateCaptureFile(rg)
+	defer expectedCaptureFile.TearDown()
+
+	// When the request is read from the capture file
+	err := ReadFromCaptureFile(expectedCaptureFile.file, 1, func(data []byte) {
+		readPayloads = append(readPayloads, Duplicate(data))
+	})
+
+	// The read request should match the original request
+	if err != nil {
+		t.Error(err)
+	} else {
+		if !expectedCaptureFile.PayloadsEqual(readPayloads) {
+			t.Error("Request read back from file should match")
+		}
+	}
+
+}
+
+func TestInputFileWithPayloadLargerThan64Kb(t *testing.T) {
+
+	input := NewTestInput()
+	rg := NewRequestGenerator([]io.Reader{input}, func() { input.EmitSizedPOST(64 * 1024) }, 1)
+	readPayloads := [][]byte{}
+
+	// Given a capture file with a request over 64Kb
+	expectedCaptureFile := CreateCaptureFile(rg)
+	defer expectedCaptureFile.TearDown()
+
+	// When the request is read from the capture file
+	err := ReadFromCaptureFile(expectedCaptureFile.file, 1, func(data []byte) {
+		readPayloads = append(readPayloads, Duplicate(data))
+	})
+
+	// The read request should match the original request
+	if err != nil {
+		t.Error(err)
+	} else {
+		if !expectedCaptureFile.PayloadsEqual(readPayloads) {
+			t.Error("Request read back from file should match")
+		}
+	}
+
+}
+
+func TestInputFileWithGETAndPOST(t *testing.T) {
+
+	input := NewTestInput()
+	rg := NewRequestGenerator([]io.Reader{input}, func() {
+		input.EmitGET()
+		input.EmitPOST()
+	}, 2)
+	readPayloads := [][]byte{}
+
+	// Given a capture file with a GET request
+	expectedCaptureFile := CreateCaptureFile(rg)
+	defer expectedCaptureFile.TearDown()
+
+	// When the requests are read from the capture file
+	err := ReadFromCaptureFile(expectedCaptureFile.file, 2, func(data []byte) {
+		readPayloads = append(readPayloads, Duplicate(data))
+	})
+
+	// The read requests should match the original request
+	if err != nil {
+		t.Error(err)
+	} else {
+		if !expectedCaptureFile.PayloadsEqual(readPayloads) {
+			t.Error("Request read back from file should match")
+		}
+	}
+
+}
+
+type CaptureFile struct {
+	data [][]byte
+	file *os.File
+}
+
+func NewExpectedCaptureFile(data [][]byte, file *os.File) *CaptureFile {
+	ecf := new(CaptureFile)
+	ecf.file = file
+	ecf.data = data
+	return ecf
+}
+
+func (expectedCaptureFile *CaptureFile) TearDown() {
+	if expectedCaptureFile.file != nil {
+		syscall.Unlink(expectedCaptureFile.file.Name())
+	}
+}
+
+type RequestGenerator struct {
+	inputs []io.Reader
+	emit   func()
+	wg     *sync.WaitGroup
+}
+
+func NewRequestGenerator(inputs []io.Reader, emit func(), count int) (rg *RequestGenerator) {
+	rg = new(RequestGenerator)
+	rg.inputs = inputs
+	rg.emit = emit
+	rg.wg = new(sync.WaitGroup)
+	rg.wg.Add(count)
+	return
+}
+
+func (expectedCaptureFile *CaptureFile) PayloadsEqual(other [][]byte) bool {
+
+	if len(expectedCaptureFile.data) != len(other) {
+		return false
+	}
+
+	for i, payload := range other {
+
+		if !bytes.Equal(expectedCaptureFile.data[i], payload) {
+			return false
+		}
+
+	}
+
+	return true
+
+}
+
+func CreateCaptureFile(requestGenerator *RequestGenerator) *CaptureFile {
+
+	f, err := ioutil.TempFile("", "testmainconf")
+	if err != nil {
+		panic(err)
+	}
+
+	quit := make(chan int)
+
+	readPayloads := [][]byte{}
+	output := NewTestOutput(func(data []byte) {
+
+		readPayloads = append(readPayloads, Duplicate(data))
+
+		requestGenerator.wg.Done()
+	})
+
+	outputFile := NewFileOutput(f.Name())
+
+	Plugins.Inputs = requestGenerator.inputs
+	Plugins.Outputs = []io.Writer{output, outputFile}
+
+	go Start(quit)
+
+	requestGenerator.emit()
+	requestGenerator.wg.Wait()
+
+	close(quit)
+
+	return NewExpectedCaptureFile(readPayloads, f)
+
+}
+
+func ReadFromCaptureFile(captureFile *os.File, count int, callback writeCallback) (err error) {
+
+	quit := make(chan int)
+	wg := new(sync.WaitGroup)
+
+	input := NewFileInput(captureFile.Name())
+	output := NewTestOutput(func(data []byte) {
+		callback(data)
+		wg.Done()
+	})
+
+	Plugins.Inputs = []io.Reader{input}
+	Plugins.Outputs = []io.Writer{output}
+
+	wg.Add(count)
+	go Start(quit)
+
+	done := make(chan int, 1)
+	go func() {
+		wg.Wait()
+		done <- 1
+	}()
+
+	select {
+	case <-done:
+		break
+	case <-time.After(2 * time.Second):
+		err = errors.New("Timed out")
+	}
+	close(quit)
+
+	return
+
+}
+
+func Duplicate(data []byte) (duplicate []byte) {
+	duplicate = make([]byte, len(data))
+	copy(duplicate, data)
+
+	return
+}
@@ -14,15 +14,23 @@ type RAWInput struct {
 	address  string
 	expire   time.Duration
 	quit     chan bool
+	engine   int
 	listener *raw.Listener
 }

+// Available engines for intercepting traffic
+const (
+	EngineRawSocket = 1 << iota
+	EnginePcap
+)
+
 // NewRAWInput constructor for RAWInput. Accepts address with port as argument.
-func NewRAWInput(address string, expire time.Duration) (i *RAWInput) {
+func NewRAWInput(address string, engine int, expire time.Duration) (i *RAWInput) {
 	i = new(RAWInput)
 	i.data = make(chan *raw.TCPMessage)
 	i.address = address
 	i.expire = expire
+	i.engine = engine
 	i.quit = make(chan bool)

 	go i.listen(address)
@@ -59,7 +67,7 @@ func (i *RAWInput) listen(address string) {
 		log.Fatal("input-raw: error while parsing address", err)
 	}

-	i.listener = raw.NewListener(host, port, i.expire)
+	i.listener = raw.NewListener(host, port, i.engine, i.expire)

 	for {
 		select {
@@ -76,7 +84,7 @@ func (i *RAWInput) listen(address string) {
 }

 func (i *RAWInput) String() string {
-	return "RAW Socket input: " + i.address
+	return "Intercepting traffic from: " + i.address
 }

 func (i *RAWInput) Close() {
@@ -5,11 +5,13 @@ import (
 	"io"
 	"io/ioutil"
 	"log"
+	"math/rand"
 	"net/http"
 	"net/http/httptest"
 	"net/http/httputil"
 	"os"
 	"os/exec"
+	"strconv"
 	"strings"
 	"sync"
 	"sync/atomic"
@@ -29,7 +31,7 @@ func TestRAWInput(t *testing.T) {

 	var respCounter, reqCounter int64

-	input := NewRAWInput(originAddr, testRawExpire)
+	input := NewRAWInput(originAddr, EnginePcap, testRawExpire)
 	defer input.Close()

 	output := NewTestOutput(func(data []byte) {
@@ -51,14 +53,14 @@ func TestRAWInput(t *testing.T) {

 	client := NewHTTPClient(origin.URL, &HTTPClientConfig{})

-	time.Sleep(time.Millisecond)
-
 	go Start(quit)
+	time.Sleep(100 * time.Millisecond)

 	for i := 0; i < 100; i++ {
 		// request + response
 		wg.Add(2)
 		client.Get("/")
+		time.Sleep(2 * time.Millisecond)
 	}

 	wg.Wait()
@@ -69,7 +71,7 @@ func TestInputRAW100Expect(t *testing.T) {
 	wg := new(sync.WaitGroup)
 	quit := make(chan int)

-	fileContent, _ := ioutil.ReadFile("LICENSE.txt")
+	fileContent, _ := ioutil.ReadFile("COMM-LICENSE")

 	// Origing and Replay server initialization
 	origin := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -81,7 +83,7 @@ func TestInputRAW100Expect(t *testing.T) {

 	originAddr := strings.Replace(origin.Listener.Addr().String(), "[::]", "127.0.0.1", -1)

-	input := NewRAWInput(originAddr, time.Second)
+	input := NewRAWInput(originAddr, EnginePcap, time.Second)
 	defer input.Close()

 	// We will use it to get content of raw HTTP request
@@ -116,10 +118,11 @@ func TestInputRAW100Expect(t *testing.T) {
 	Plugins.Outputs = []io.Writer{testOutput, httpOutput}

 	go Start(quit)
+	time.Sleep(100 * time.Millisecond)

 	// Origin + Response/Request Test Output + Request Http Output
 	wg.Add(4)
-	curl := exec.Command("curl", "http://"+originAddr, "--data-binary", "@LICENSE.txt")
+	curl := exec.Command("curl", "http://"+originAddr, "--data-binary", "@COMM-LICENSE")
 	err := curl.Run()
 	if err != nil {
 		log.Fatal(err)
@@ -144,7 +147,7 @@ func TestInputRAWChunkedEncoding(t *testing.T) {
 	}))

 	originAddr := strings.Replace(origin.Listener.Addr().String(), "[::]", "127.0.0.1", -1)
-	input := NewRAWInput(originAddr, time.Second)
+	input := NewRAWInput(originAddr, EnginePcap, time.Second)
 	defer input.Close()

 	replay := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -166,10 +169,11 @@ func TestInputRAWChunkedEncoding(t *testing.T) {
 	Plugins.Outputs = []io.Writer{httpOutput}

 	go Start(quit)
+	time.Sleep(100 * time.Millisecond)

 	wg.Add(2)

-	curl := exec.Command("curl", "http://"+originAddr, "--header", "Transfer-Encoding: chunked", "--data-binary", "@README.md")
+	curl := exec.Command("curl", "http://"+originAddr, "--header", "Transfer-Encoding: chunked", "--header", "Expect:", "--data-binary", "@README.md")
 	err := curl.Run()
 	if err != nil {
 		log.Fatal(err)
@@ -187,9 +191,10 @@ func TestInputRAWLargePayload(t *testing.T) {
 	}
 	wg := new(sync.WaitGroup)
 	quit := make(chan int)
+	sizeKb := 100

 	// Generate 100kb file
-	dd := exec.Command("dd", "if=/dev/urandom", "of=/tmp/large", "bs=1KB", "count=100")
+	dd := exec.Command("dd", "if=/dev/urandom", "of=/tmp/large", "bs=1KB", "count="+strconv.Itoa(sizeKb))
 	err := dd.Run()
 	if err != nil {
 		log.Fatal("dd error:", err)
@@ -199,7 +204,7 @@ func TestInputRAWLargePayload(t *testing.T) {
 		defer req.Body.Close()
 		body, _ := ioutil.ReadAll(req.Body)

-		if len(body) != 100*1000 {
+		if len(body) != sizeKb*1000 {
 			t.Error("File size should be 1mb:", len(body))
 		}

@@ -207,17 +212,18 @@ func TestInputRAWLargePayload(t *testing.T) {
 	}))
 	originAddr := strings.Replace(origin.Listener.Addr().String(), "[::]", "127.0.0.1", -1)

-	input := NewRAWInput(originAddr, time.Second)
+	input := NewRAWInput(originAddr, EnginePcap, testRawExpire)
 	defer input.Close()

 	replay := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
-		req.Body = http.MaxBytesReader(w, req.Body, 1*1024*1024)
-		buf := make([]byte, 1*1024*1024)
-		n, _ := req.Body.Read(buf)
-		body := buf[0:n]
+		body, _ := ioutil.ReadAll(req.Body)
+		// // req.Body = http.MaxBytesReader(w, req.Body, 1*1024*1024)
+		// // buf := make([]byte, 1*1024*1024)
+		// n, _ := req.Body.Read(buf)
+		// body := buf[0:n]

-		if len(body) != 100*1000 {
-			t.Error("File size should be 100000 bytes:", len(body))
+		if len(body) != sizeKb*1000 {
+			t.Errorf("File size should be %d bytes: %d", sizeKb*1000, len(body))
 		}

 		wg.Done()
@@ -231,8 +237,10 @@ func TestInputRAWLargePayload(t *testing.T) {

 	go Start(quit)

+	time.Sleep(100 * time.Millisecond)
+
 	wg.Add(2)
-	curl := exec.Command("curl", "http://"+originAddr, "--header", "Transfer-Encoding: chunked", "--data-binary", "@/tmp/large")
+	curl := exec.Command("curl", "http://"+originAddr, "--header", "Transfer-Encoding: chunked", "--header", "Expect:", "--data-binary", "@/tmp/large")
 	err = curl.Run()
 	if err != nil {
 		log.Fatal("curl error:", err)
@@ -241,3 +249,62 @@ func TestInputRAWLargePayload(t *testing.T) {
 	wg.Wait()
 	close(quit)
 }
+
+func BenchmarkRAWInput(b *testing.B) {
+	quit := make(chan int)
+
+	origin := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
+	defer origin.Close()
+	originAddr := strings.Replace(origin.Listener.Addr().String(), "[::]", "127.0.0.1", -1)
+
+	var respCounter, reqCounter int64
+
+	input := NewRAWInput(originAddr, EnginePcap, testRawExpire)
+	defer input.Close()
+
+	output := NewTestOutput(func(data []byte) {
+		if data[0] == '1' {
+			atomic.AddInt64(&reqCounter, 1)
+		} else {
+			atomic.AddInt64(&respCounter, 1)
+		}
+
+		// log.Println("Captured ", reqCounter, "requests and ", respCounter, " responses")
+	})
+
+	Plugins.Inputs = []io.Reader{input}
+	Plugins.Outputs = []io.Writer{output}
+
+	time.Sleep(time.Millisecond)
+
+	go Start(quit)
+
+	emitted := 0
+	fileContent, _ := ioutil.ReadFile("LICENSE.txt")
+
+	for i := 0; i < b.N; i++ {
+		wg := new(sync.WaitGroup)
+		wg.Add(10 * 100)
+		emitted += 10 * 100
+		for w := 0; w < 100; w++ {
+			go func() {
+				client := NewHTTPClient(origin.URL, &HTTPClientConfig{})
+				for i := 0; i < 10; i++ {
+					if rand.Int63n(2) == 0 {
+						client.Post("/", fileContent)
+					} else {
+						client.Get("/")
+					}
+					time.Sleep(time.Duration(rand.Int63n(50)) * time.Millisecond)
+					wg.Done()
+				}
+			}()
+		}
+		wg.Wait()
+	}
+
+	time.Sleep(400 * time.Millisecond)
+	log.Println("Emitted ", emitted, ", Captured ", reqCounter, "requests and ", respCounter, " responses")
+
+	close(quit)
+}
@@ -85,13 +85,22 @@ func (m *Middleware) copy(to io.Writer, from io.Reader) {
 }

 func (m *Middleware) read(from io.Reader) {
-	scanner := bufio.NewScanner(from)
+	reader := bufio.NewReader(from)
+	var line []byte
+	var e error

-	for scanner.Scan() {
-		bytes := scanner.Bytes()
-		buf := make([]byte, len(bytes)/2)
-		if _, err := hex.Decode(buf, bytes); err != nil {
-			fmt.Fprintln(os.Stderr, "Failed to decode input payload", err, len(bytes))
+	for {
+		if line, e = reader.ReadBytes('\n'); e != nil {
+			if e == io.EOF {
+				continue
+			} else {
+				break
+			}
+		}
+
+		buf := make([]byte, len(line)/2)
+		if _, err := hex.Decode(buf, line[:len(line)-1]); err != nil {
+			fmt.Fprintln(os.Stderr, "Failed to decode input payload", err, len(line))
 		}

 		if Settings.debug {
@@ -101,10 +110,6 @@ func (m *Middleware) read(from io.Reader) {
 		m.data <- buf
 	}

-	if err := scanner.Err(); err != nil {
-		fmt.Fprintln(os.Stderr, "Traffic modifier command failed:", err)
-	}
-
 	return
 }

@@ -116,7 +116,8 @@ func TestEchoMiddleware(t *testing.T) {
 	Settings.middleware = "./examples/middleware/echo.sh"

 	// Catch traffic from one service
-	input := NewRAWInput(from.Listener.Addr().String(), testRawExpire)
+	fromAddr := strings.Replace(from.Listener.Addr().String(), "[::]", "127.0.0.1", -1)
+	input := NewRAWInput(fromAddr, EnginePcap, testRawExpire)
 	defer input.Close()

 	// And redirect to another
@@ -129,7 +130,7 @@ func TestEchoMiddleware(t *testing.T) {
 	go Start(quit)

 	// Wait till middleware initialization
-	time.Sleep(10*time.Millisecond)
+	time.Sleep(100 * time.Millisecond)

 	// Should receive 2 requests from original + 2 from replayed
 	client := NewHTTPClient(from.URL, &HTTPClientConfig{Debug: false})
@@ -138,7 +139,9 @@ func TestEchoMiddleware(t *testing.T) {
 		wg.Add(4)
 		// Request should be echoed
 		client.Get("/a")
+		time.Sleep(5 * time.Millisecond)
 		client.Get("/b")
+		time.Sleep(5 * time.Millisecond)
 	}

 	wg.Wait()
@@ -176,7 +179,7 @@ func TestTokenMiddleware(t *testing.T) {

 	fromAddr := strings.Replace(from.Listener.Addr().String(), "[::]", "127.0.0.1", -1)
 	// Catch traffic from one service
-	input := NewRAWInput(fromAddr, testRawExpire)
+	input := NewRAWInput(fromAddr, EnginePcap, testRawExpire)
 	defer input.Close()

 	// And redirect to another
@@ -50,7 +50,7 @@ func (o *TCPOutput) worker() {
 		_, err := conn.Write([]byte(payloadSeparator))

 		if err != nil {
-			log.Println("Worker failed on write, exitings and starting new worker")
+			log.Println("Worker failed on write, exitings and starting new worker:", err)
 			go o.worker()
 			break
 		}
@@ -79,8 +79,13 @@ func InitPlugins() {
 		registerPlugin(NewDummyOutput, options)
 	}

+	engine := EnginePcap
+	if Settings.inputRAWEngine == "raw_socket" {
+		engine = EngineRawSocket
+	}
+
 	for _, options := range Settings.inputRAW {
-		registerPlugin(NewRAWInput, options, time.Duration(0))
+		registerPlugin(NewRAWInput, options, engine, time.Duration(0))
 	}

 	for _, options := range Settings.inputTCP {
@@ -15,16 +15,25 @@ package rawSocket
 import (
 	"bytes"
 	"encoding/binary"
+	"fmt"
+	"github.com/google/gopacket"
+	_ "github.com/google/gopacket/layers"
+	"github.com/google/gopacket/pcap"
+	"io"
 	"log"
 	"net"
 	"runtime/debug"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 )

+var _ = fmt.Println
+
 // Listener handle traffic capture
 type Listener struct {
+	mu sync.Mutex
 	// buffer of TCPMessages waiting to be send
 	// ID -> TCPMessage
 	messages map[string]*TCPMessage
@@ -57,12 +66,19 @@ type Listener struct {
 }

 type request struct {
+	id    string
 	start time.Time
 	ack   uint32
 }

+// Available engines for intercepting traffic
+const (
+	EngineRawSocket = 1 << iota
+	EnginePcap
+)
+
 // NewListener creates and initializes new Listener object
-func NewListener(addr string, port string, expire time.Duration) (l *Listener) {
+func NewListener(addr string, port string, engine int, expire time.Duration) (l *Listener) {
 	l = &Listener{}

 	l.packetsChan = make(chan *TCPPacket, 10000)
@@ -86,15 +102,34 @@ func NewListener(addr string, port string, expire time.Duration) (l *Listener) {
 	l.messageExpire = expire

 	go l.listen()
+	go l.processPackets()

 	// Special case for testing
 	if l.port != 0 {
-		go l.readRAWSocket()
+		switch engine {
+		case EngineRawSocket:
+			go l.readRAWSocket()
+		case EnginePcap:
+			go l.readPcap()
+		default:
+			log.Fatal("Unknown traffic interception engine:", engine)
+		}
 	}

 	return
 }

+func (t *Listener) processPackets() {
+	for {
+		// We need to use channels to process each packet to avoid data races
+		packet := <-t.packetsChan
+		// log.Println(packet)
+		t.mu.Lock()
+		t.processTCPPacket(packet)
+		t.mu.Unlock()
+	}
+}
+
 func (t *Listener) listen() {
 	gcTicker := time.Tick(t.messageExpire / 2)

@@ -105,36 +140,41 @@ func (t *Listener) listen() {
 				t.conn.Close()
 			}
 			return
-		// We need to use channels to process each packet to avoid data races
-		case packet := <-t.packetsChan:
-			maxLen := len(packet.Data)
-			if maxLen > 500 {
-				maxLen = 500
-			}
-
-			t.processTCPPacket(packet)
-
 		case <-gcTicker:
 			now := time.Now()
+			// log.Println("GC")

+			t.mu.Lock()
+			// Dispatch requests before responses
 			for _, message := range t.messages {
-				if now.Sub(message.Start) >= t.messageExpire {
+				if now.Sub(message.End) >= t.messageExpire {
 					t.dispatchMessage(message)
 				}
 			}
+
+			t.mu.Unlock()
 		}
 	}
 }

 func (t *Listener) dispatchMessage(message *TCPMessage) {
+	// If already dispatched
+	if _, ok := t.messages[message.ID]; !ok {
+		return
+	}
+
 	delete(t.ackAliases, message.Ack)
 	delete(t.messages, message.ID)

+	// log.Println("Dispatching, message", message.Seq, message.Ack, string(message.Bytes()))
+
 	if message.IsIncoming {
 		// If there were response before request
+		// log.Println("Looking for Response: ", t.respWithoutReq, message.ResponseAck)
 		if respID, ok := t.respWithoutReq[message.ResponseAck]; ok {
 			if resp, rok := t.messages[respID]; rok {
 				if resp.RequestAck == 0 {
+					// log.Println("FOUND RESPONSE")
 					resp.RequestAck = message.Ack
 					resp.RequestStart = message.Start

@@ -145,11 +185,19 @@ func (t *Listener) dispatchMessage(message *TCPMessage) {
 			}
 		}
 	} else {
+		if message.RequestAck == 0 {
+			if responseRequest, ok := t.respAliases[message.Ack]; ok {
+				message.RequestStart = responseRequest.start
+				message.RequestAck = responseRequest.ack
+			}
+		}
+
 		delete(t.respAliases, message.Ack)
 		delete(t.respWithoutReq, message.Ack)

 		// Do not track responses which have no associated requests
 		if message.RequestAck == 0 {
+			// log.Println("Can't dispatch resp", message.Seq, message.Ack, string(message.Bytes()))
 			return
 		}
 	}
@@ -157,6 +205,113 @@ func (t *Listener) dispatchMessage(message *TCPMessage) {
 	t.messagesChan <- message
 }

+// DeviceNotFoundError raised if user specified wrong ip
+type DeviceNotFoundError struct {
+	addr string
+}
+
+func (e *DeviceNotFoundError) Error() string {
+	devices, _ := pcap.FindAllDevs()
+
+	var msg string
+	msg += "Devices with addr: " + e.addr + " not found. Available devices: \n"
+	for _, device := range devices {
+		msg += "Name: " + device.Name + "\n"
+		msg += "Description: " + device.Description + "\n"
+		msg += "Devices addresses: " + device.Description + "\n"
+		for _, address := range device.Addresses {
+			msg += "- IP address: " + address.IP.String() + "\n"
+			msg += "- Subnet mask: " + address.Netmask.String() + "\n"
+		}
+	}
+
+	return msg
+}
+
+func findPcapDevice(addr string) (*pcap.Interface, error) {
+	devices, err := pcap.FindAllDevs()
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	for _, device := range devices {
+		if device.Name == "any" && addr == "" || addr == "0.0.0.0" {
+			return &device, nil
+		}
+
+		for _, address := range device.Addresses {
+			if address.IP.String() == addr {
+				return &device, nil
+			}
+		}
+	}
+
+	return nil, &DeviceNotFoundError{addr}
+}
+
+func (t *Listener) readPcap() {
+	device, err := findPcapDevice(t.addr)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	handle, err := pcap.OpenLive(device.Name, 65536, true, t.messageExpire)
+	if err != nil {
+		log.Fatal(err)
+	}
+	defer handle.Close()
+
+	if err := handle.SetBPFFilter("tcp and port " + strconv.Itoa(int(t.port))); err != nil {
+		log.Fatal(err)
+	}
+
+	source := gopacket.NewPacketSource(handle, handle.LinkType())
+	source.Lazy = true
+	source.NoCopy = true
+
+	// log.Println(handle.Stats())
+
+	for {
+		packet, err := source.NextPacket()
+
+		if err == io.EOF {
+			break
+		} else if err != nil {
+			continue
+		}
+
+		// Skip ethernet layer, 14 bytes
+		data := packet.Data()[14:]
+		ihl := uint8(data[0]) & 0x0F
+
+		// Truncated IP info
+		if len(data) < int(ihl*4) {
+			continue
+		}
+
+		srcIP := data[12:16]
+		data = data[ihl*4:]
+
+		// Truncated TCP info
+		if len(data) < 13 {
+			continue
+		}
+
+		dataOffset := (data[12] & 0xF0) >> 4
+
+		// We need only packets with data inside
+		// Check that the buffer is larger than the size of the TCP header
+		if len(data) > int(dataOffset*4) {
+			newBuf := make([]byte, len(data))
+			copy(newBuf, data)
+
+			go func(newBuf []byte) {
+				t.packetsChan <- ParseTCPPacket(net.IP(srcIP).String(), newBuf)
+			}(newBuf)
+		}
+	}
+}
+
 func (t *Listener) readRAWSocket() {
 	conn, e := net.ListenPacket("ip4:tcp", t.addr)
 	t.conn = conn
@@ -187,7 +342,7 @@ func (t *Listener) readRAWSocket() {
 				copy(newBuf, buf[:n])

 				go func(newBuf []byte) {
-					t.packetsChan <- ParseTCPPacket(addr, newBuf)
+					t.packetsChan <- ParseTCPPacket(addr.String(), newBuf)
 				}(newBuf)
 			}
 		}
@@ -230,14 +385,32 @@ func (t *Listener) processTCPPacket(packet *TCPPacket) {
 		}
 	}()

+	// log.Println("Processing packet:", packet.Ack, packet.Seq, string(packet.Data))
+
 	var message *TCPMessage

 	isIncoming := packet.DestPort == t.port

+	// Seek for 100-expect chunks
 	if parentAck, ok := t.seqWithData[packet.Seq]; ok {
+		// log.Println("Found data package with Ack:", packet.Ack)
+		// In case if non-first data chunks comes first
+		for _id, m := range t.messages {
+			// log.Println("Message ack:", m.Ack, m.packets[0].Addr, packet.Addr)
+			if m.Ack == packet.Ack && m.packets[0].Addr == packet.Addr {
+				delete(t.messages, _id)
+
+				for _, pkt := range m.packets {
+					pkt.Ack = parentAck
+					// Re-queue this packets
+					t.processTCPPacket(pkt)
+				}
+			}
+		}
+
+		delete(t.seqWithData, packet.Seq)
 		t.ackAliases[packet.Ack] = parentAck
 		packet.Ack = parentAck
-		delete(t.seqWithData, packet.Seq)
 	}

 	if alias, ok := t.ackAliases[packet.Ack]; ok {
@@ -250,7 +423,7 @@ func (t *Listener) processTCPPacket(packet *TCPPacket) {
 		responseRequest, _ = t.respAliases[packet.Ack]
 	}

-	mID := packet.Addr.String() + strconv.Itoa(int(packet.DestPort)) + strconv.Itoa(int(packet.Ack))
+	mID := packet.Addr + strconv.Itoa(int(packet.DestPort)) + strconv.Itoa(int(packet.Ack))

 	message, ok := t.messages[mID]

@@ -262,6 +435,7 @@ func (t *Listener) processTCPPacket(packet *TCPPacket) {
 			if responseRequest != nil {
 				message.RequestStart = responseRequest.start
 				message.RequestAck = responseRequest.ack
+				message.RequestID = responseRequest.id
 			} else {
 				t.respWithoutReq[packet.Ack] = mID
 			}
@@ -276,7 +450,9 @@ func (t *Listener) processTCPPacket(packet *TCPPacket) {
 			t.seqWithData[seq] = packet.Ack

 			// In case if sequence packet came first
+			// log.Println("Looking for sequences:", seq, t.messages)
 			for _id, m := range t.messages {
+				// log.Println("SeqSEQ", m.Seq, len(m.packets))
 				if m.Seq == seq {
 					t.ackAliases[m.Ack] = packet.Ack

@@ -290,27 +466,41 @@ func (t *Listener) processTCPPacket(packet *TCPPacket) {

 			// Removing `Expect: 100-continue` header
 			packet.Data = append(packet.Data[:len(packet.Data)-24], packet.Data[len(packet.Data)-2:]...)
+
+			// log.Println(string(packet.Data))
 		}
 	}

-	if isIncoming {
-		// If message have multiple packets, delete previous alias
-		if len(message.packets) > 0 {
-			delete(t.respAliases, message.ResponseAck)
-		}
-
-		responseAck := packet.Seq + uint32(message.BodySize()) + uint32(len(packet.Data))
-		t.respAliases[responseAck] = &request{message.Start, message.Ack}
-
-		message.ResponseAck = responseAck
-	}
-
 	// Adding packet to message
 	message.AddPacket(packet)

+	if isIncoming {
+		// If message have multiple packets, delete previous alias
+		if len(message.packets) > 1 {
+			delete(t.respAliases, message.ResponseAck)
+		}
+
+		message.UpdateResponseAck()
+		t.respAliases[message.ResponseAck] = &request{message.ID, message.Start, message.Ack}
+	}
+
 	// If message contains only single packet immediately dispatch it
 	if message.IsFinished() {
-		t.dispatchMessage(message)
+		if isIncoming {
+			if resp, ok := t.messages[message.ResponseID()]; ok {
+				t.dispatchMessage(message)
+				if resp.IsFinished() {
+					t.dispatchMessage(resp)
+				}
+			}
+		} else {
+			if req, ok := t.messages[message.RequestID]; ok {
+				if req.IsFinished() {
+					t.dispatchMessage(req)
+					t.dispatchMessage(message)
+				}
+			}
+		}
 	}
 }

@@ -10,21 +10,21 @@ import (
 func TestRawListenerInput(t *testing.T) {
 	var req, resp *TCPMessage

-	listener := NewListener("", "0", 10*time.Millisecond)
+	listener := NewListener("", "0", EnginePcap, 10*time.Millisecond)
 	defer listener.Close()

 	reqPacket := buildPacket(true, 1, 1, []byte("GET / HTTP/1.1"))

-	listener.packetsChan <- reqPacket
-
 	respAck := reqPacket.Seq + uint32(len(reqPacket.Data))
 	respPacket := buildPacket(false, respAck, reqPacket.Seq+1, []byte("HTTP/1.1 200 OK"))
-	listener.packetsChan <- respPacket
+
+	listener.processTCPPacket(reqPacket)
+	listener.processTCPPacket(respPacket)

 	select {
 	case req = <-listener.messagesChan:
 	case <-time.After(time.Millisecond):
-		t.Error("Should return respose immediately")
+		t.Error("Should return request immediately")
 		return
 	}

@@ -34,7 +34,7 @@ func TestRawListenerInput(t *testing.T) {

 	select {
 	case resp = <-listener.messagesChan:
-	case <-time.After(time.Millisecond):
+	case <-time.After(20 * time.Millisecond):
 		t.Error("Should return response immediately")
 		return
 	}
@@ -47,15 +47,15 @@ func TestRawListenerInput(t *testing.T) {
 func TestRawListenerResponse(t *testing.T) {
 	var req, resp *TCPMessage

-	listener := NewListener("", "0", 10*time.Millisecond)
+	listener := NewListener("", "0", EnginePcap, 10*time.Millisecond)
 	defer listener.Close()

 	reqPacket := buildPacket(true, 1, 1, []byte("GET / HTTP/1.1"))
 	respPacket := buildPacket(false, 1+uint32(len(reqPacket.Data)), 2, []byte("HTTP/1.1 200 OK"))

 	// If response packet comes before request
-	listener.packetsChan <- respPacket
-	listener.packetsChan <- reqPacket
+	listener.processTCPPacket(respPacket)
+	listener.processTCPPacket(reqPacket)

 	select {
 	case req = <-listener.messagesChan:
@@ -87,7 +87,7 @@ func TestRawListenerResponse(t *testing.T) {
 func TestRawListener100Continue(t *testing.T) {
 	var req, resp *TCPMessage

-	listener := NewListener("", "0", 10*time.Millisecond)
+	listener := NewListener("", "0", EnginePcap, 10*time.Millisecond)
 	defer listener.Close()

 	reqPacket1 := buildPacket(true, 1, 1, []byte("POST / HTTP/1.1\r\nContent-Length: 2\r\nExpect: 100-continue\r\n\r\n"))
@@ -98,7 +98,7 @@ func TestRawListener100Continue(t *testing.T) {
 	respPacket1 := buildPacket(false, 10, 3, []byte("HTTP/1.1 100 Continue\r\n"))

 	// panic(int(uint32(len(reqPacket1.Data)) + uint32(len(reqPacket2.Data)) + uint32(len(reqPacket3.Data))))
-	respPacket2 := buildPacket(false, reqPacket3.Seq+2 /* len of data */, 2, []byte("HTTP/1.1 200 OK\r\n"))
+	respPacket2 := buildPacket(false, reqPacket3.Seq+1 /* len of data */, 2, []byte("HTTP/1.1 200 OK\r\n"))

 	listener.processTCPPacket(reqPacket1)
 	listener.processTCPPacket(reqPacket2)
@@ -107,6 +107,63 @@ func TestRawListener100Continue(t *testing.T) {
 	listener.processTCPPacket(respPacket1)
 	listener.processTCPPacket(respPacket2)

+	select {
+	case req = <-listener.messagesChan:
+		break
+	case <-time.After(11 * time.Millisecond):
+		t.Error("Should return request after expire time")
+		return
+	}
+
+	if !bytes.Equal(req.Bytes(), []byte("POST / HTTP/1.1\r\nContent-Length: 2\r\n\r\nab")) {
+		t.Error("Should receive full message", string(req.Bytes()))
+	}
+
+	if !req.IsIncoming {
+		t.Error("Should be request")
+	}
+
+	select {
+	case resp = <-listener.messagesChan:
+		break
+	case <-time.After(21 * time.Millisecond):
+		t.Error("Should return response after expire time")
+		return
+	}
+
+	if resp.IsIncoming {
+		t.Error("Should be response")
+	}
+
+	if !bytes.Equal(resp.UUID(), req.UUID()) {
+		t.Error("Resp and Req UUID should be equal")
+	}
+}
+
+// Response comes before Request
+func TestRawListener100ContinueWrongOrder(t *testing.T) {
+	var req, resp *TCPMessage
+
+	listener := NewListener("", "0", EnginePcap, 10*time.Millisecond)
+	defer listener.Close()
+
+	reqPacket1 := buildPacket(true, 1, 1, []byte("POST / HTTP/1.1\r\nContent-Length: 2\r\nExpect: 100-continue\r\n\r\n"))
+	// Packet with data have different Seq
+	reqPacket2 := buildPacket(true, 2, reqPacket1.Seq+uint32(len(reqPacket1.Data)), []byte("a"))
+	reqPacket3 := buildPacket(true, 2, reqPacket2.Seq+1, []byte("b"))
+
+	respPacket1 := buildPacket(false, 10, 3, []byte("HTTP/1.1 100 Continue\r\n"))
+
+	// panic(int(uint32(len(reqPacket1.Data)) + uint32(len(reqPacket2.Data)) + uint32(len(reqPacket3.Data))))
+	respPacket2 := buildPacket(false, reqPacket3.Seq+1 /* len of data */, 2, []byte("HTTP/1.1 200 OK\r\n"))
+
+	listener.processTCPPacket(respPacket1)
+	listener.processTCPPacket(respPacket2)
+
+	listener.processTCPPacket(reqPacket1)
+	listener.processTCPPacket(reqPacket2)
+	listener.processTCPPacket(reqPacket3)
+
 	select {
 	case req = <-listener.messagesChan:
 		break
@@ -140,59 +197,95 @@ func TestRawListener100Continue(t *testing.T) {
 	}
 }

-// Response comes before Request
-func TestRawListener100ContinueWrongOrder(t *testing.T) {
-    var req, resp *TCPMessage
+func testChunkedSequence(t *testing.T, listener *Listener, packets ...*TCPPacket) {
+	var r, req, resp *TCPMessage

-    listener := NewListener("", "0", 10*time.Millisecond)
-    defer listener.Close()
+	for _, p := range packets {
+		listener.processTCPPacket(p)
+	}

-    reqPacket1 := buildPacket(true, 1, 1, []byte("POST / HTTP/1.1\r\nContent-Length: 2\r\nExpect: 100-continue\r\n\r\n"))
-    // Packet with data have different Seq
-    reqPacket2 := buildPacket(true, 2, reqPacket1.Seq+uint32(len(reqPacket1.Data)), []byte("a"))
-    reqPacket3 := buildPacket(true, 2, reqPacket2.Seq+1, []byte("b"))
+	select {
+	case r = <-listener.messagesChan:
+		if r.IsIncoming {
+			req = r
+		} else {
+			resp = r
+		}
+		break
+	case <-time.After(25 * time.Millisecond):
+		t.Error("Should return request after expire time")
+		return
+	}
+	select {
+	case r = <-listener.messagesChan:
+		if r.IsIncoming {
+			req = r
+		} else {
+			resp = r
+		}
+		break
+	case <-time.After(25 * time.Millisecond):
+		t.Error("Should return request after expire time")
+		return
+	}

-    respPacket1 := buildPacket(false, 10, 3, []byte("HTTP/1.1 100 Continue\r\n"))
+	if !bytes.Equal(req.Bytes(), []byte("POST / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n1\r\na\r\n1\r\nb\r\n0\r\n\r\n")) {
+		t.Error("Should receive full message", string(req.Bytes()))
+	}

-    // panic(int(uint32(len(reqPacket1.Data)) + uint32(len(reqPacket2.Data)) + uint32(len(reqPacket3.Data))))
-    respPacket2 := buildPacket(false, reqPacket3.Seq+2 /* len of data */, 2, []byte("HTTP/1.1 200 OK\r\n"))
+	if !req.IsIncoming {
+		t.Error("Should be request")
+	}

-    listener.processTCPPacket(respPacket1)
-    listener.processTCPPacket(respPacket2)
+	if resp.IsIncoming {
+		t.Error("Should be response")
+	}

-    listener.processTCPPacket(reqPacket1)
-    listener.processTCPPacket(reqPacket2)
-    listener.processTCPPacket(reqPacket3)
+	if !bytes.Equal(resp.UUID(), req.UUID()) {
+		t.Error("Resp and Req UUID should be equal", string(resp.UUID()), string(req.UUID()))
+	}

-    select {
-    case req = <-listener.messagesChan:
-        break
-    case <-time.After(11 * time.Millisecond):
-        t.Error("Should return response after expire time")
-        return
-    }
+	time.Sleep(15 * time.Millisecond)

-    if !bytes.Equal(req.Bytes(), []byte("POST / HTTP/1.1\r\nContent-Length: 2\r\n\r\nab")) {
-        t.Error("Should receive full message", string(req.Bytes()))
-    }
-
-    if !req.IsIncoming {
-        t.Error("Should be request")
-    }
-
-    select {
-    case resp = <-listener.messagesChan:
-        break
-    case <-time.After(21 * time.Millisecond):
-        t.Error("Should return response after expire time")
-        return
-    }
-
-    if resp.IsIncoming {
-        t.Error("Should be response")
-    }
-
-    if !bytes.Equal(resp.UUID(), req.UUID()) {
-        t.Error("Resp and Req UUID should be equal")
-    }
+	if len(listener.messages) != 0 {
+		t.Error("Messages non empty:", listener.messages)
+	}
+}
+
+func permutation(n int, list []*TCPPacket) []*TCPPacket {
+	if len(list) == 1 {
+		return list
+	}
+
+	k := n % len(list)
+
+	first := []*TCPPacket{list[k]}
+	next := make([]*TCPPacket, len(list)-1)
+
+	copy(next, append(list[:k], list[k+1:]...))
+
+	return append(first, permutation(n/len(list), next)...)
+}
+
+// Response comes before Request
+func TestRawListenerChunkedWrongOrder(t *testing.T) {
+	listener := NewListener("", "0", EnginePcap, 10*time.Millisecond)
+	defer listener.Close()
+
+	reqPacket1 := buildPacket(true, 1, 1, []byte("POST / HTTP/1.1\r\nTransfer-Encoding: chunked\r\nExpect: 100-continue\r\n\r\n"))
+	// Packet with data have different Seq
+	reqPacket2 := buildPacket(true, 2, reqPacket1.Seq+uint32(len(reqPacket1.Data)), []byte("1\r\na\r\n"))
+	reqPacket3 := buildPacket(true, 2, reqPacket2.Seq+uint32(len(reqPacket2.Data)), []byte("1\r\nb\r\n"))
+	reqPacket4 := buildPacket(true, 2, reqPacket3.Seq+uint32(len(reqPacket3.Data)), []byte("0\r\n\r\n"))
+
+	respPacket1 := buildPacket(false, 10, 3, []byte("HTTP/1.1 100 Continue\r\n"))
+
+	// panic(int(uint32(len(reqPacket1.Data)) + uint32(len(reqPacket2.Data)) + uint32(len(reqPacket3.Data))))
+	respPacket2 := buildPacket(false, reqPacket4.Seq+5 /* len of data */, 2, []byte("HTTP/1.1 200 OK\r\n"))
+
+	// Should re-construct message from all possible combinations
+	for i := 0; i < 6*5*4*3*2*1; i++ {
+		packets := permutation(i, []*TCPPacket{reqPacket1, reqPacket2, reqPacket3, reqPacket4, respPacket1, respPacket2})
+		testChunkedSequence(t, listener, packets...)
+	}
 }
@@ -23,6 +23,7 @@ type TCPMessage struct {
 	ResponseAck  uint32
 	RequestStart time.Time
 	RequestAck   uint32
+	RequestID    string
 	Start        time.Time
 	End          time.Time
 	IsIncoming   bool
@@ -95,11 +96,23 @@ func (t *TCPMessage) AddPacket(packet *TCPPacket) {
 		// Packets not always captured in same Seq order, and sometimes we need to prepend
 		if len(t.packets) == 0 || packet.Seq > t.packets[len(t.packets)-1].Seq {
 			t.packets = append(t.packets, packet)
-		} else {
+		} else if packet.Seq < t.packets[0].Seq {
 			t.packets = append([]*TCPPacket{packet}, t.packets...)
+			t.Seq = packet.Seq // Message Seq should indicated starting seq
+		} else { // insert somewhere in the middle...
+			for i, p := range t.packets {
+				if packet.Seq < p.Seq {
+					t.packets = append(t.packets[:i], append([]*TCPPacket{packet}, t.packets[i:]...)...)
+					break
+				}
+			}
 		}

-		t.End = time.Now()
+		if t.IsIncoming {
+			t.End = time.Now()
+		} else {
+			t.End = time.Now().Add(time.Millisecond)
+		}
 	}
 }

@@ -175,3 +188,15 @@ func (t *TCPMessage) UUID() []byte {

 	return uuid
 }
+
+// UpdateResponseAck should be called after packet is added
+func (t *TCPMessage) UpdateResponseAck() uint32 {
+	lastPacket := t.packets[len(t.packets)-1]
+	t.ResponseAck = lastPacket.Seq + uint32(len(lastPacket.Data))
+	return t.ResponseAck
+}
+
+// ResponseID generate message ID for request response
+func (t *TCPMessage) ResponseID() string {
+	return t.packets[0].Addr + strconv.Itoa(int(t.packets[0].SrcPort)) + strconv.Itoa(int(t.ResponseAck))
+}
@@ -3,14 +3,13 @@ package rawSocket
 import (
 	"bytes"
 	_ "log"
-	"net"
 	"strconv"
 	"testing"
 )

 func buildPacket(isIncoming bool, Ack, Seq uint32, Data []byte) (packet *TCPPacket) {
 	packet = &TCPPacket{
-		Addr: &net.IPAddr{net.IP{}, ""},
+		Addr: "",
 		Ack:  Ack,
 		Seq:  Seq,
 		Data: Data,
@@ -27,7 +26,7 @@ func buildPacket(isIncoming bool, Ack, Seq uint32, Data []byte) (packet *TCPPack
 }

 func buildMessage(p *TCPPacket) *TCPMessage {
-	id := p.Addr.String() + strconv.Itoa(int(p.DestPort)) + strconv.Itoa(int(p.Ack))
+	id := p.Addr + strconv.Itoa(int(p.DestPort)) + strconv.Itoa(int(p.Ack))

 	isIncoming := false
 	if p.SrcPort == 1 {
@@ -2,7 +2,6 @@ package rawSocket

 import (
 	"encoding/binary"
-	"net"
 	"strconv"
 	"strings"
 )
@@ -35,11 +34,11 @@ type TCPPacket struct {

 	Data []byte

-	Addr net.Addr
+	Addr string
 }

 // ParseTCPPacket takes address and tcp payload and returns parsed TCPPacket
-func ParseTCPPacket(addr net.Addr, b []byte) (p *TCPPacket) {
+func ParseTCPPacket(addr string, b []byte) (p *TCPPacket) {
 	p = &TCPPacket{Data: b}
 	p.ParseBasic()
 	p.Addr = addr
@@ -75,7 +74,7 @@ func (t *TCPPacket) String() string {
 	}

 	return strings.Join([]string{
-		"Addr: " + t.Addr.String(),
+		"Addr: " + t.Addr,
 		"Source port: " + strconv.Itoa(int(t.SrcPort)),
 		"Dest port:" + strconv.Itoa(int(t.DestPort)),
 		"Sequence:" + strconv.Itoa(int(t.Seq)),
@@ -41,7 +41,8 @@ type AppSettings struct {
 	inputFile  MultiOption
 	outputFile MultiOption

-	inputRAW MultiOption
+	inputRAW       MultiOption
+	inputRAWEngine string

 	middleware string

@@ -82,6 +83,8 @@ func init() {

 	flag.Var(&Settings.inputRAW, "input-raw", "Capture traffic from given port (use RAW sockets and require *sudo* access):\n\t# Capture traffic from 8080 port\n\tgor --input-raw :8080 --output-http staging.com")

+	flag.StringVar(&Settings.inputRAWEngine, "input-raw-engine", "libpcap", "Intercept traffic using `libpcap` (default), and `raw_socket`")
+
 	flag.StringVar(&Settings.middleware, "middleware", "", "Used for modifying traffic using external command")

 	flag.Var(&Settings.inputHTTP, "input-http", "Read requests from HTTP, should be explicitly sent from your application:\n\t# Listen for http on 9000\n\tgor --input-http :9000 --output-http staging.com")
@@ -93,6 +96,7 @@ func init() {

 	flag.BoolVar(&Settings.outputHTTPConfig.stats, "output-http-stats", false, "Report http output queue stats to console every 5 seconds.")
 	flag.BoolVar(&Settings.outputHTTPConfig.OriginalHost, "http-original-host", false, "Normally gor replaces the Host http header with the host supplied with --output-http.  This option disables that behavior, preserving the original Host header.")
+	flag.BoolVar(&Settings.outputHTTPConfig.Debug, "output-http-debug", false, "Enables http debug output.")

 	flag.StringVar(&Settings.outputHTTPConfig.elasticSearch, "output-http-elasticsearch", "", "Send request and response stats to ElasticSearch:\n\tgor --input-raw :8080 --output-http staging.com --output-http-elasticsearch 'es_host:api_port/index_name'")

@@ -53,6 +53,18 @@ func (i *TestInput) EmitLargePOST() {
 	rs := base64.URLEncoding.EncodeToString(rb)

 	i.data <- []byte("POST / HTTP/1.1\nHost: www.w3.org\nContent-Length:5242880\r\n\r\n" + rs)
+	Debug("Sent large POST")
+}
+
+// EmitSizedPOST emit a POST with a payload set to a supplied size
+func (i *TestInput) EmitSizedPOST(payloadSize int) {
+	rb := make([]byte, payloadSize)
+	rand.Read(rb)
+
+	rs := base64.URLEncoding.EncodeToString(rb)
+
+	i.data <- []byte("POST / HTTP/1.1\nHost: www.w3.org\nContent-Length:5242880\r\n\r\n" + rs)
+	Debug("Sent large POST")
 }

 // EmitOPTIONS emits OPTIONS request, similar to GET
Author	SHA1	Message	Date
Leonid Bugaev	281219e6a0	Use static linking for libpcap and fix error with truncated packets	2016-05-03 17:55:38 +05:00
Leonid Bugaev	ba227ae9ba	Fix license file	2016-05-03 10:28:15 +05:00
Leonid Bugaev	5779ea1b45	Update java middleware to not use apache commons	2016-05-02 16:31:38 +05:00
Leonid Bugaev	bb0fab4d35	Update README.md	2016-04-30 18:35:51 +05:00
Leonid Bugaev	8cc377f22f	Update README.md	2016-04-30 18:35:13 +05:00
Leonid Bugaev	5bea65c723	Fix input raw command line options	2016-04-29 21:44:50 +05:00
Leonid Bugaev	52a0ebfd57	Fmt fixes	2016-04-29 16:01:37 +05:00
Leonid Bugaev	a8731f290c	Fix broken pipe error	2016-04-29 16:01:30 +05:00
Leonid Bugaev	2f0e865e82	Fix middleware crashing for big payloads. Fix #245 and #250	2016-04-29 15:34:38 +05:00
Leonid Bugaev	afc8f7852e	Libpcap support and vairous bugfixes Merge #260	2016-04-29 14:48:17 +05:00
Leonid Bugaev	00feb08165	Add benchmark for testing packet capture (and its quality)	2016-04-26 17:18:09 +05:00
Leonid Bugaev	0754ed7d83	Merge pull request #239 from dougjohnson/output-http-debug add flag to enable debugging of http output	2016-04-25 17:55:04 +06:00
Leonid Bugaev	4baf3985dc	More debugging	2016-04-25 16:46:34 +05:00
Leonid Bugaev	32264eb427	Update README.md	2016-04-25 10:44:58 +05:00
Salimane Adjao Moustapha	f2cedaf89f	Merge pull request #256 from buger/revert-238-fix-memory-leak-in-aliases Revert "Fixes memory leak by adding respAliases cleanup after expiration time"	2016-04-23 21:35:38 +02:00
Salimane Adjao Moustapha	e8f7d30551	Revert "Fixes memory leak by adding respAliases cleanup after expiration time"	2016-04-23 21:17:54 +02:00
Salimane Adjao Moustapha	0c7643f477	Merge pull request #238 from AnderEnder/fix-memory-leak-in-aliases Fixes memory leak by adding respAliases cleanup after expiration time	2016-04-23 20:23:11 +02:00
Salimane Adjao Moustapha	4c8a73a58b	Merge pull request #235 from sneakybeaky/input_file_reader Replace the use of bufio.Scanner with bufio.Reader	2016-04-23 20:20:42 +02:00
Jon Barber	e659f65d76	Replace the use of bufio.Scanner with bufio.Reader so that requests over 64Kb in size can be replayed from a capture file.	2016-04-21 10:15:48 +01:00
Andrii Radyk	7a2f1aef4c	Merge remote-tracking branch 'upstream/master' into fix-memory-leak-in-aliases	2016-04-21 09:23:03 +02:00
Doug Johnson	699eefe317	add flag to enable debugging of http output	2016-01-26 21:56:55 +00:00
Andrii Radyk	2a955dcdbf	Fixes memory leak by adding respAliases cleanup after expiration time	2016-01-26 13:26:47 +01:00